2024-09-22T01:00:01.511 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-22T01:00:01.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T01:00:01.605 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914605' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writeback global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: stupid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKV5b2WKvQmx8ptEvimAe8dS8jSsPqoNYlaLLOzFyqNe9zJxjKyLFOfbJHqwLkaRbeHcg/kfkqmiE9sh3JJQtts= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDOeOlqVTw6+W0xSuoRg+T2rxt2lEo4rz8gHxYTeTJHFle2XCKzhMNq9Z8gTNzYotitNw2QP/VJh7CZ3VmFjOYg= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-22T01:00:01.605 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-22T01:00:01.606 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-22T01:00:01.606 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-22T01:00:01.607 INFO:teuthology.task.internal:Checking packages... 2024-09-22T01:00:01.624 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-22T01:00:01.625 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-22T01:00:01.625 INFO:teuthology.packaging:ref: None 2024-09-22T01:00:01.625 INFO:teuthology.packaging:tag: None 2024-09-22T01:00:01.625 INFO:teuthology.packaging:branch: squid 2024-09-22T01:00:01.625 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T01:00:01.625 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-22T01:00:01.839 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:00:01.840 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-22T01:00:01.847 INFO:teuthology.task.internal:no buildpackages task found 2024-09-22T01:00:01.847 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-22T01:00:01.853 INFO:teuthology.task.internal:Saving configuration 2024-09-22T01:00:01.866 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-22T01:00:01.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-22T01:00:01.898 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 00:54:13.343980', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIOzNGxcy5GWso1Pr9g7OL2U4TI4p9dyqnpj/KccRCZb'} 2024-09-22T01:00:01.919 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-22 00:54:13.345920', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKgxdd+BlJ99v3+ofYflURdMwQ+Kql03wQQdEWDH16+H'} 2024-09-22T01:00:01.919 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-22T01:00:01.925 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-22T01:00:01.925 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-22T01:00:01.925 INFO:teuthology.run_tasks:Running task console_log... 2024-09-22T01:00:01.986 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6712b67880>, signals=[15]) 2024-09-22T01:00:01.986 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-22T01:00:01.991 INFO:teuthology.task.internal:Opening connections... 2024-09-22T01:00:01.992 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:00:01.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:00:02.070 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:00:02.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:00:02.148 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-22T01:00:02.155 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-22T01:00:02.160 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-22T01:00:02.160 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-22T01:00:02.208 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-22T01:00:02.208 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2024-09-22T01:00:02.208 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="22.04" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=jammy 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-22T01:00:02.209 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=jammy 2024-09-22T01:00:02.210 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-22T01:00:02.233 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-22T01:00:02.238 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-22T01:00:02.239 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Ubuntu" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="22.04" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_CODENAME=jammy 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:ID=ubuntu 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE=debian 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-22T01:00:02.284 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-22T01:00:02.285 INFO:teuthology.orchestra.run.smithi150.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-22T01:00:02.285 INFO:teuthology.orchestra.run.smithi150.stdout:UBUNTU_CODENAME=jammy 2024-09-22T01:00:02.285 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-22T01:00:02.308 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-22T01:00:02.317 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-22T01:00:02.335 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-22T01:00:02.335 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-22T01:00:02.337 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-22T01:00:02.342 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-22T01:00:02.347 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-22T01:00:02.347 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-22T01:00:02.381 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-22T01:00:02.404 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-22T01:00:02.455 INFO:teuthology.run_tasks:Running task kernel... 2024-09-22T01:00:02.468 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-22T01:00:02.468 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-22T01:00:02.468 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T01:00:02.468 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-22T01:00:02.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:00:02.473 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-22T01:00:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:00:02.474 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-22T01:00:02.521 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-56-generic 2024-09-22T01:00:02.521 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-56-generic 2024-09-22T01:00:02.521 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.15.0-56-generic 2024-09-22T01:00:02.522 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2024-09-22T01:00:02.522 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.15.0-56-generic 2024-09-22T01:00:02.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-22T01:00:02.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-22T01:00:02.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-22T01:00:04.274 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-22T01:00:04.326 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:00:04.369 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-22T01:00:04.426 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-22T01:00:04.599 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-22T01:00:04.603 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-22T01:00:04.698 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-22T01:00:04.712 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-22T01:00:04.716 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-22T01:00:04.717 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-22T01:00:04.742 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-22T01:00:04.743 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-22T01:00:04.744 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-22T01:00:04.748 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-22T01:00:04.748 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-22T01:00:04.751 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-22T01:00:04.760 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-22T01:00:04.767 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-22T01:00:04.770 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-22T01:00:04.771 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-22T01:00:04.771 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-22T01:00:04.772 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-22T01:00:04.805 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-22T01:00:04.805 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-22T01:00:04.805 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-22T01:00:04.806 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-22T01:00:04.806 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-22T01:00:04.806 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-22T01:00:04.806 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-22T01:00:04.807 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-22T01:00:04.821 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-22T01:00:04.847 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-22T01:00:04.858 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-22T01:00:04.864 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-22T01:00:04.865 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-22T01:00:04.901 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:00:04.901 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-22T01:00:04.902 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-22T01:00:04.906 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-22T01:00:04.906 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-22T01:00:04.912 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-22T01:00:04.918 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-22T01:00:04.919 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-22T01:00:04.921 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-22T01:00:04.921 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-22T01:00:04.921 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-22T01:00:04.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-22T01:00:04.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-22T01:00:04.994 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-22T01:00:05.037 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-22T01:00:05.076 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [703 kB] 2024-09-22T01:00:05.141 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,062 kB] 2024-09-22T01:00:05.176 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [356 kB] 2024-09-22T01:00:05.180 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-22T01:00:05.180 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,499 kB] 2024-09-22T01:00:05.199 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-22T01:00:05.199 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [430 kB] 2024-09-22T01:00:05.202 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-22T01:00:05.203 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [733 kB] 2024-09-22T01:00:05.208 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,125 kB] 2024-09-22T01:00:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-22T01:00:05.256 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-22T01:00:05.258 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-22T01:00:05.258 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-22T01:00:05.259 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-22T01:00:05.259 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-22T01:00:05.259 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-22T01:00:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-22T01:00:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-22T01:00:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-22T01:00:05.263 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-22T01:00:05.264 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-22T01:00:05.274 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-22T01:00:05.291 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-22T01:00:05.292 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-22T01:00:05.292 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-22T01:00:05.292 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-22T01:00:05.332 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-22T01:00:05.333 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-22T01:00:05.341 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-22T01:00:05.342 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [629 kB] 2024-09-22T01:00:05.354 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [905 kB] 2024-09-22T01:00:05.371 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-22T01:00:05.377 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-22T01:00:05.424 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-22T01:00:05.425 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-22T01:00:05.426 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-22T01:00:05.427 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-22T01:00:11.147 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 16.3 MB in 4s (4,354 kB/s) 2024-09-22T01:00:11.348 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 16.3 MB in 4s (3,945 kB/s) 2024-09-22T01:00:12.707 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:00:12.727 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T01:00:12.791 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:00:12.807 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:00:12.824 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T01:00:12.879 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:00:13.003 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-22T01:00:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-22T01:00:13.065 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-22T01:00:13.066 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-22T01:00:13.181 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:00:13.181 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:00:13.181 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:00:13.182 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:00:13.182 INFO:teuthology.orchestra.run.smithi096.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-22T01:00:13.182 INFO:teuthology.orchestra.run.smithi096.stdout: python2.7-minimal 2024-09-22T01:00:13.182 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:00:13.183 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-22T01:00:13.183 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T01:00:13.183 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-22T01:00:13.183 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-22T01:00:13.185 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-22T01:00:13.185 INFO:teuthology.orchestra.run.smithi096.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-22T01:00:13.185 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-22T01:00:13.185 INFO:teuthology.orchestra.run.smithi096.stdout: thermald 2024-09-22T01:00:13.235 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:00:13.235 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:00:13.235 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:00:13.235 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:00:13.235 INFO:teuthology.orchestra.run.smithi150.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-22T01:00:13.236 INFO:teuthology.orchestra.run.smithi150.stdout: python2.7-minimal 2024-09-22T01:00:13.236 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:00:13.237 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2024-09-22T01:00:13.237 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T01:00:13.237 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-22T01:00:13.237 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-22T01:00:13.238 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2024-09-22T01:00:13.239 INFO:teuthology.orchestra.run.smithi150.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-22T01:00:13.239 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2024-09-22T01:00:13.239 INFO:teuthology.orchestra.run.smithi150.stdout: thermald 2024-09-22T01:00:13.245 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-22T01:00:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T01:00:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-22T01:00:13.246 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-22T01:00:13.247 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-22T01:00:13.248 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-22T01:00:13.298 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2024-09-22T01:00:13.298 INFO:teuthology.orchestra.run.smithi150.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-22T01:00:13.298 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-22T01:00:13.299 INFO:teuthology.orchestra.run.smithi150.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-22T01:00:13.300 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages will be upgraded: 2024-09-22T01:00:13.301 INFO:teuthology.orchestra.run.smithi150.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-22T01:00:13.427 INFO:teuthology.orchestra.run.smithi096.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T01:00:13.427 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 113 MB of archives. 2024-09-22T01:00:13.427 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-22T01:00:13.428 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-22T01:00:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:3 upgraded, 5 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T01:00:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 113 MB of archives. 2024-09-22T01:00:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-22T01:00:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-22T01:00:14.003 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-22T01:00:14.106 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-22T01:00:14.782 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-22T01:00:14.908 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-22T01:00:14.909 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-22T01:00:14.909 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-22T01:00:15.030 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-22T01:00:15.052 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-22T01:00:15.133 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-22T01:00:15.391 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 113 MB in 2s (64.9 MB/s) 2024-09-22T01:00:15.546 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-22T01:00:17.351 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-22T01:00:17.351 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-22T01:00:17.352 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-22T01:00:17.560 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-22T01:00:17.564 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:17.626 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:17.696 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-22T01:00:17.775 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-22T01:00:18.178 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 113 MB in 5s (25.1 MB/s) 2024-09-22T01:00:18.391 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-22T01:00:20.286 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-22T01:00:20.307 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:20.384 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:20.464 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-22T01:00:20.468 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:20.555 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:20.856 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-22T01:00:20.876 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:20.914 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:22.699 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-22T01:00:22.720 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:22.802 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:23.336 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-22T01:00:23.357 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:23.411 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:27.198 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:27.317 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:27.607 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:27.729 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:28.032 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-22T01:00:28.054 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-22T01:00:28.098 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T01:00:30.588 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:30.840 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:31.257 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:31.386 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:31.697 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-22T01:00:31.720 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-22T01:00:31.755 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T01:00:33.920 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-22T01:00:33.957 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:33.994 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:36.552 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:36.673 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:36.990 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T01:00:37.111 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:37.229 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-22T01:00:37.282 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-22T01:00:37.319 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-22T01:00:37.357 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:37.358 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:38.728 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:00:38.728 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-22T01:00:38.728 INFO:teuthology.orchestra.run.smithi150.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:00:38.728 INFO:teuthology.orchestra.run.smithi150.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-22T01:00:39.293 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:40.661 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-22T01:00:40.762 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-22T01:00:40.888 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:41.733 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-22T01:00:41.858 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-22T01:00:42.157 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-22T01:00:42.273 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:42.321 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:42.334 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-22T01:00:42.334 INFO:teuthology.orchestra.run.smithi150.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:00:42.447 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-22T01:00:42.601 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:43.928 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:00:43.928 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-22T01:00:43.928 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:00:43.928 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-22T01:00:44.114 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:45.796 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-22T01:00:45.921 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-22T01:00:46.067 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:47.457 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-22T01:00:47.515 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-22T01:00:47.515 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:00:55.975 INFO:teuthology.orchestra.run.smithi150.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-22T01:00:55.975 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub' 2024-09-22T01:00:55.991 INFO:teuthology.orchestra.run.smithi150.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T01:00:56.014 INFO:teuthology.orchestra.run.smithi150.stdout:Generating grub configuration file ... 2024-09-22T01:00:56.294 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:00:56.321 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:00:56.633 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:00:56.639 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T01:00:56.711 INFO:teuthology.orchestra.run.smithi150.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T01:00:56.717 INFO:teuthology.orchestra.run.smithi150.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T01:00:56.871 INFO:teuthology.orchestra.run.smithi150.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T01:00:56.871 INFO:teuthology.orchestra.run.smithi150.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T01:00:56.871 INFO:teuthology.orchestra.run.smithi150.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T01:00:56.917 INFO:teuthology.orchestra.run.smithi150.stdout:done 2024-09-22T01:00:57.450 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 21 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.122.122 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T01:00:57.470 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T01:00:57.471 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2024-09-22T01:00:57.471 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2024-09-22T01:00:57.471 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2024-09-22T01:00:57.471 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2024-09-22T01:00:57.471 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-22T01:00:57.471 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T01:00:57.471 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T01:00:57.471 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-22T01:00:57.471 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T01:00:57.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get clean 2024-09-22T01:00:57.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-22T01:00:57.757 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T01:00:57.876 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:00:57.960 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T01:00:58.048 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T01:00:59.211 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:00:59.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T01:00:59.281 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:00:59.467 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-22T01:00:59.468 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-22T01:00:59.636 INFO:teuthology.orchestra.run.smithi150.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-22T01:00:59.636 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:00:59.636 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:00:59.637 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:00:59.637 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:00:59.637 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T01:00:59.637 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T01:00:59.637 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:00:59.676 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T01:00:59.678 DEBUG:teuthology.orchestra.run.smithi150:> dpkg -s linux-image-generic 2024-09-22T01:00:59.734 INFO:teuthology.orchestra.run.smithi150.stdout:Package: linux-image-generic 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Status: install ok installed 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Priority: optional 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Section: kernel 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Installed-Size: 21 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: amd64 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Source: linux-meta 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Version: 5.15.0.122.122 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T01:00:59.735 INFO:teuthology.orchestra.run.smithi150.stdout:Recommends: thermald 2024-09-22T01:00:59.736 INFO:teuthology.orchestra.run.smithi150.stdout:Description: Generic Linux kernel image 2024-09-22T01:00:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: This package will always depend on the latest generic kernel image 2024-09-22T01:00:59.736 INFO:teuthology.orchestra.run.smithi150.stdout: available. 2024-09-22T01:00:59.736 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-22T01:00:59.780 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.fYvq0pYVXT 2024-09-22T01:00:59.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub/grub.cfg /tmp/tmp.fYvq0pYVXT 2024-09-22T01:00:59.836 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.fYvq0pYVXT 2024-09-22T01:00:59.957 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.fYvq0pYVXT is 10KB 2024-09-22T01:00:59.970 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.fYvq0pYVXT 2024-09-22T01:00:59.975 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-22T01:01:00.028 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:01:00.028 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-22T01:01:00.028 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-22T01:01:00.095 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-22T01:01:00.096 DEBUG:teuthology.orchestra.run.smithi150:> sudo update-grub 2024-09-22T01:01:01.307 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub' 2024-09-22T01:01:01.308 INFO:teuthology.orchestra.run.smithi150.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T01:01:01.310 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-22T01:01:01.494 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:01:01.500 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:01:01.750 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:01:01.756 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T01:01:01.799 INFO:teuthology.orchestra.run.smithi150.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T01:01:01.805 INFO:teuthology.orchestra.run.smithi150.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T01:01:01.850 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-22T01:01:01.850 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub' 2024-09-22T01:01:01.876 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T01:01:01.904 INFO:teuthology.orchestra.run.smithi096.stdout:Generating grub configuration file ... 2024-09-22T01:01:01.936 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T01:01:01.936 INFO:teuthology.orchestra.run.smithi150.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T01:01:01.936 INFO:teuthology.orchestra.run.smithi150.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T01:01:01.956 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-22T01:01:01.965 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-22T01:01:02.253 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:01:02.270 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:01:02.634 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:01:02.639 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T01:01:02.704 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T01:01:02.710 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T01:01:02.874 INFO:teuthology.orchestra.run.smithi096.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T01:01:02.874 INFO:teuthology.orchestra.run.smithi096.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T01:01:02.874 INFO:teuthology.orchestra.run.smithi096.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T01:01:02.912 INFO:teuthology.orchestra.run.smithi096.stdout:done 2024-09-22T01:01:03.387 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-22T01:01:03.405 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.122.122 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-22T01:01:03.406 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-22T01:01:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-22T01:01:03.407 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-22T01:01:03.407 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-22T01:01:03.407 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-22T01:01:03.407 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-22T01:01:03.407 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-22T01:01:03.407 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-09-22T01:01:03.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-22T01:01:03.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-22T01:01:03.693 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T01:01:03.832 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:01:03.912 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T01:01:03.994 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T01:01:05.166 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:01:05.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-22T01:01:05.237 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:01:05.425 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-22T01:01:05.425 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:01:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T01:01:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T01:01:05.595 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:01:05.636 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 0 newly installed, 0 to remove and 345 not upgraded. 2024-09-22T01:01:05.637 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-22T01:01:05.694 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.122.122 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-22T01:01:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-22T01:01:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-22T01:01:05.696 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-22T01:01:05.739 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.YQDwOYc4xF 2024-09-22T01:01:05.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub/grub.cfg /tmp/tmp.YQDwOYc4xF 2024-09-22T01:01:05.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.YQDwOYc4xF 2024-09-22T01:01:05.919 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.YQDwOYc4xF is 10KB 2024-09-22T01:01:05.933 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.YQDwOYc4xF 2024-09-22T01:01:05.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-22T01:01:05.990 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:01:05.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-22T01:01:05.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-22T01:01:06.054 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-22T01:01:06.054 DEBUG:teuthology.orchestra.run.smithi096:> sudo update-grub 2024-09-22T01:01:07.432 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub' 2024-09-22T01:01:07.433 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-22T01:01:07.437 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-09-22T01:01:07.627 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-22T01:01:07.632 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-22T01:01:07.941 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-22T01:01:07.947 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-22T01:01:07.986 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-22T01:01:07.991 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-22T01:01:08.111 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-22T01:01:08.111 INFO:teuthology.orchestra.run.smithi096.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-22T01:01:08.111 INFO:teuthology.orchestra.run.smithi096.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-22T01:01:08.134 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-09-22T01:01:08.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-09-22T01:01:31.992 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-22T01:01:31.992 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:01:31.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:01:38.154 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-22T01:01:38.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:01:38.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:01:50.447 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-22T01:01:56.591 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-22T01:01:59.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:01:59.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:02:02.515 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-22T01:02:05.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:02:05.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:02:08.655 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-22T01:02:14.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:02:14.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:02:20.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:02:20.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:02:52.332 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-22T01:02:52.887 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:02:52.888 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-22T01:02:52.888 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-22T01:02:52.934 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-122-generic 2024-09-22T01:02:52.934 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-22T01:02:52.935 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T01:02:52.935 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-22T01:02:53.936 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-22T01:02:53.936 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T01:02:54.012 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-09-22T01:02:54.032 DEBUG:teuthology.parallel:result is None 2024-09-22T01:03:14.521 DEBUG:teuthology.orchestra.remote:timed out 2024-09-22T01:03:29.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:03:29.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:03:29.836 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-22T01:03:30.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:03:30.362 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-22T01:03:30.362 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-22T01:03:30.407 INFO:teuthology.orchestra.run.smithi150.stdout:5.15.0-122-generic 2024-09-22T01:03:30.407 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-22T01:03:30.407 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-22T01:03:30.407 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-22T01:03:31.408 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-22T01:03:31.408 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-22T01:03:31.767 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-09-22T01:03:31.797 DEBUG:teuthology.parallel:result is None 2024-09-22T01:03:31.798 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-22T01:03:31.810 INFO:teuthology.task.internal:Creating test directory... 2024-09-22T01:03:31.810 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T01:03:31.813 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-22T01:03:31.818 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-22T01:03:31.828 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-22T01:03:31.836 INFO:teuthology.task.internal:Creating archive directory... 2024-09-22T01:03:31.836 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T01:03:31.859 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-22T01:03:31.943 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-22T01:03:31.949 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-22T01:03:31.949 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T01:03:31.952 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-22T01:03:31.971 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T01:03:31.974 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T01:03:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T01:03:31.982 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-22T01:03:31.983 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-22T01:03:31.991 INFO:teuthology.task.internal:Configuring sudo... 2024-09-22T01:03:31.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T01:03:32.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-22T01:03:32.040 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-22T01:03:32.049 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-22T01:03:32.049 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T01:03:32.083 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-22T01:03:32.088 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T01:03:32.134 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T01:03:32.182 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:03:32.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T01:03:32.243 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-22T01:03:32.249 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-22T01:03:32.295 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:03:32.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-22T01:03:32.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-22T01:03:32.361 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-22T01:03:32.437 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-22T01:03:32.445 INFO:teuthology.task.internal:Starting timer... 2024-09-22T01:03:32.445 INFO:teuthology.run_tasks:Running task pcp... 2024-09-22T01:03:32.482 INFO:teuthology.run_tasks:Running task selinux... 2024-09-22T01:03:32.507 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2024-09-22T01:03:32.508 DEBUG:teuthology.task.selinux:Excluding smithi150: OS 'ubuntu' does not support SELinux 2024-09-22T01:03:32.508 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-22T01:03:32.508 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-22T01:03:32.508 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-22T01:03:32.508 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-22T01:03:32.518 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-22T01:03:32.528 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-22T01:03:32.529 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-22T01:10:59.988 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-09-22T01:10:59.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:10:59.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:11:00.068 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-22T01:11:00.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-22T01:11:00.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:11:00.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-22T01:11:00.224 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-22T01:11:00.299 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-22T01:11:00.299 INFO:teuthology.run_tasks:Running task clock... 2024-09-22T01:11:00.318 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-22T01:11:00.318 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T01:11:00.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:11:00.321 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-22T01:11:00.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Command line: ntpd -gq 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: ---------------------------------------------------- 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: ntp-4 is maintained by Network Time Foundation, 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-22T01:11:00.349 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: corporation. Support and training for ntp-4 are 2024-09-22T01:11:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: available at https://www.nwtime.org/support 2024-09-22T01:11:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: ---------------------------------------------------- 2024-09-22T01:11:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: proto: precision = 0.059 usec (-24) 2024-09-22T01:11:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: basedate set to 2022-02-04 2024-09-22T01:11:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: gps base set to 2022-02-06 (week 2196) 2024-09-22T01:11:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T01:11:00.351 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T01:11:00.351 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: restrict ::: KOD does nothing without LIMITED. 2024-09-22T01:11:00.351 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-22T01:11:00.352 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen and drop on 0 v6wildcard [::]:123 2024-09-22T01:11:00.352 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-22T01:11:00.352 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen normally on 2 lo 127.0.0.1:123 2024-09-22T01:11:00.352 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2024-09-22T01:11:00.352 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen normally on 4 lo [::1]:123 2024-09-22T01:11:00.353 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2024-09-22T01:11:00.353 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:00 ntpd[18480]: Listening on routing socket on fd #22 for interface updates 2024-09-22T01:11:00.366 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Command line: ntpd -gq 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: ---------------------------------------------------- 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: ntp-4 is maintained by Network Time Foundation, 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: corporation. Support and training for ntp-4 are 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: available at https://www.nwtime.org/support 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: ---------------------------------------------------- 2024-09-22T01:11:00.367 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: proto: precision = 0.061 usec (-24) 2024-09-22T01:11:00.370 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: basedate set to 2022-02-04 2024-09-22T01:11:00.370 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: gps base set to 2022-02-06 (week 2196) 2024-09-22T01:11:00.370 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T01:11:00.370 INFO:teuthology.orchestra.run.smithi150.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-22T01:11:00.370 INFO:teuthology.orchestra.run.smithi150.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: restrict ::: KOD does nothing without LIMITED. 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen and drop on 0 v6wildcard [::]:123 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen normally on 2 lo 127.0.0.1:123 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen normally on 3 enp3s0f1 172.21.15.150:123 2024-09-22T01:11:00.371 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen normally on 4 lo [::1]:123 2024-09-22T01:11:00.372 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e35%5]:123 2024-09-22T01:11:00.372 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:00 ntpd[18490]: Listening on routing socket on fd #22 for interface updates 2024-09-22T01:11:01.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:01.352 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:01 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:01.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:01.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:01.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:01.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:01 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:03.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:03.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:03 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:03.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:03.367 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:03.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:03.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:03 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.352 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.352 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.353 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.354 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.355 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.355 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.355 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:05 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:05.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:05 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:07.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.350 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stdout:22 Sep 01:11:07 ntpd[18480]: ntpd: time slew -0.001147 s 2024-09-22T01:11:07.351 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.001147s 2024-09-22T01:11:07.352 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-22T01:11:07.352 INFO:teuthology.orchestra.run.smithi096.stderr:22 Sep 01:11:07 ntpd[18480]: can't open /var/log/ntpstats/loopstats.20240922: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.368 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/rawstats.20240922: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/peerstats.20240922: Permission denied 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stdout:22 Sep 01:11:07 ntpd[18490]: ntpd: time slew +0.000369 s 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stdout:ntpd: time slew +0.000369s 2024-09-22T01:11:07.369 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-22T01:11:07.370 INFO:teuthology.orchestra.run.smithi150.stderr:22 Sep 01:11:07 ntpd[18490]: can't open /var/log/ntpstats/loopstats.20240922: Permission denied 2024-09-22T01:11:07.416 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:11:07.417 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2024-09-22T01:11:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.417 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.465 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:11:07.465 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-22T01:11:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.466 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:11:07.467 INFO:teuthology.run_tasks:Running task install... 2024-09-22T01:11:07.475 DEBUG:teuthology.task.install:project ceph 2024-09-22T01:11:07.475 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T01:11:07.475 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-22T01:11:07.475 INFO:teuthology.task.install:Using flavor: default 2024-09-22T01:11:07.481 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-22T01:11:07.481 INFO:teuthology.task.install:extra packages: [] 2024-09-22T01:11:07.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2024-09-22T01:11:07.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-key list | grep Ceph 2024-09-22T01:11:07.543 INFO:teuthology.orchestra.run.smithi150.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-22T01:11:07.567 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-22T01:11:07.570 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-22T01:11:07.571 INFO:teuthology.orchestra.run.smithi150.stdout:uid [ unknown] Ceph.com (release key) 2024-09-22T01:11:07.571 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-22T01:11:07.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T01:11:07.596 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-22T01:11:07.596 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2024-09-22T01:11:07.597 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-22T01:11:07.597 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T01:11:07.725 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-22T01:11:07.725 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:11:07.743 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default/ 2024-09-22T01:11:07.743 INFO:teuthology.task.install.deb:Package version is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:11:07.839 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:11:07.839 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-22T01:11:07.853 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:11:07.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-22T01:11:07.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo apt-get update 2024-09-22T01:11:07.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-22T01:11:08.022 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:11:08.032 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T01:11:08.035 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T01:11:08.035 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-22T01:11:08.044 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-22T01:11:08.049 INFO:teuthology.orchestra.run.smithi150.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T01:11:08.061 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-22T01:11:08.076 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-22T01:11:08.089 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy InRelease 2024-09-22T01:11:08.097 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-22T01:11:08.110 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-22T01:11:08.119 INFO:teuthology.orchestra.run.smithi150.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-22T01:11:08.132 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-22T01:11:08.140 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-22T01:11:08.153 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-22T01:11:08.163 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-22T01:11:08.167 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-22T01:11:08.180 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,413 B] 2024-09-22T01:11:08.369 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 28.9 kB in 0s (86.3 kB/s) 2024-09-22T01:11:08.392 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 28.9 kB in 0s (85.4 kB/s) 2024-09-22T01:11:09.371 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:11:09.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:11:09.452 INFO:teuthology.orchestra.run.smithi150.stdout:Reading package lists... 2024-09-22T01:11:09.455 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:11:09.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-373-g4e45e6b4-1jammy cephadm=19.1.1-373-g4e45e6b4-1jammy ceph-mds=19.1.1-373-g4e45e6b4-1jammy ceph-mgr=19.1.1-373-g4e45e6b4-1jammy ceph-common=19.1.1-373-g4e45e6b4-1jammy ceph-fuse=19.1.1-373-g4e45e6b4-1jammy ceph-test=19.1.1-373-g4e45e6b4-1jammy ceph-volume=19.1.1-373-g4e45e6b4-1jammy radosgw=19.1.1-373-g4e45e6b4-1jammy python3-rados=19.1.1-373-g4e45e6b4-1jammy python3-rgw=19.1.1-373-g4e45e6b4-1jammy python3-cephfs=19.1.1-373-g4e45e6b4-1jammy python3-rbd=19.1.1-373-g4e45e6b4-1jammy libcephfs2=19.1.1-373-g4e45e6b4-1jammy libcephfs-dev=19.1.1-373-g4e45e6b4-1jammy librados2=19.1.1-373-g4e45e6b4-1jammy librbd1=19.1.1-373-g4e45e6b4-1jammy rbd-fuse=19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:11:09.535 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-22T01:11:09.662 INFO:teuthology.orchestra.run.smithi150.stdout:Building dependency tree... 2024-09-22T01:11:09.663 INFO:teuthology.orchestra.run.smithi150.stdout:Reading state information... 2024-09-22T01:11:09.744 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-22T01:11:09.745 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:11:09.861 INFO:teuthology.orchestra.run.smithi150.stdout:The following additional packages will be installed: 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T01:11:09.862 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-22T01:11:09.863 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet zip 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout:Suggested packages: 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout:Recommended packages: 2024-09-22T01:11:09.864 INFO:teuthology.orchestra.run.smithi150.stdout: btrfs-tools 2024-09-22T01:11:09.939 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-22T01:11:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-22T01:11:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-22T01:11:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-22T01:11:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-22T01:11:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-22T01:11:09.940 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-22T01:11:09.941 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-22T01:11:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-22T01:11:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-22T01:11:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-22T01:11:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T01:11:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet zip 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-22T01:11:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-22T01:11:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: btrfs-tools 2024-09-22T01:11:09.964 INFO:teuthology.orchestra.run.smithi150.stdout:The following NEW packages will be installed: 2024-09-22T01:11:09.964 INFO:teuthology.orchestra.run.smithi150.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-22T01:11:09.964 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-22T01:11:09.964 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-22T01:11:09.966 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-22T01:11:09.967 INFO:teuthology.orchestra.run.smithi150.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-22T01:11:10.027 INFO:teuthology.orchestra.run.smithi150.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-22T01:11:10.028 INFO:teuthology.orchestra.run.smithi150.stdout:Need to get 181 MB of archives. 2024-09-22T01:11:10.028 INFO:teuthology.orchestra.run.smithi150.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-22T01:11:10.028 INFO:teuthology.orchestra.run.smithi150.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-22T01:11:10.047 INFO:teuthology.orchestra.run.smithi096.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-22T01:11:10.048 INFO:teuthology.orchestra.run.smithi096.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-22T01:11:10.048 INFO:teuthology.orchestra.run.smithi096.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-22T01:11:10.049 INFO:teuthology.orchestra.run.smithi096.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-22T01:11:10.050 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-22T01:11:10.050 INFO:teuthology.orchestra.run.smithi096.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-22T01:11:10.050 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-22T01:11:10.050 INFO:teuthology.orchestra.run.smithi096.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-22T01:11:10.110 INFO:teuthology.orchestra.run.smithi150.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-22T01:11:10.118 INFO:teuthology.orchestra.run.smithi150.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-22T01:11:10.120 INFO:teuthology.orchestra.run.smithi150.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-22T01:11:10.138 INFO:teuthology.orchestra.run.smithi150.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-22T01:11:10.192 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 118 newly installed, 0 to remove and 332 not upgraded. 2024-09-22T01:11:10.193 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 181 MB of archives. 2024-09-22T01:11:10.193 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 830 MB of additional disk space will be used. 2024-09-22T01:11:10.193 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-373-g4e45e6b4-1jammy [3,592 kB] 2024-09-22T01:11:10.198 INFO:teuthology.orchestra.run.smithi150.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-22T01:11:10.203 INFO:teuthology.orchestra.run.smithi150.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-22T01:11:10.215 INFO:teuthology.orchestra.run.smithi150.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-22T01:11:10.219 INFO:teuthology.orchestra.run.smithi150.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-22T01:11:10.220 INFO:teuthology.orchestra.run.smithi150.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-22T01:11:10.221 INFO:teuthology.orchestra.run.smithi150.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-22T01:11:10.229 INFO:teuthology.orchestra.run.smithi150.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-22T01:11:10.231 INFO:teuthology.orchestra.run.smithi150.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-22T01:11:10.234 INFO:teuthology.orchestra.run.smithi150.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-22T01:11:10.235 INFO:teuthology.orchestra.run.smithi150.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-22T01:11:10.238 INFO:teuthology.orchestra.run.smithi150.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-22T01:11:10.239 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-22T01:11:10.240 INFO:teuthology.orchestra.run.smithi150.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-22T01:11:10.254 INFO:teuthology.orchestra.run.smithi150.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-22T01:11:10.255 INFO:teuthology.orchestra.run.smithi150.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-22T01:11:10.256 INFO:teuthology.orchestra.run.smithi150.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-22T01:11:10.256 INFO:teuthology.orchestra.run.smithi150.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-22T01:11:10.257 INFO:teuthology.orchestra.run.smithi150.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-22T01:11:10.257 INFO:teuthology.orchestra.run.smithi150.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-22T01:11:10.258 INFO:teuthology.orchestra.run.smithi150.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-22T01:11:10.258 INFO:teuthology.orchestra.run.smithi150.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-22T01:11:10.273 INFO:teuthology.orchestra.run.smithi150.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-22T01:11:10.274 INFO:teuthology.orchestra.run.smithi150.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-22T01:11:10.291 INFO:teuthology.orchestra.run.smithi150.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-22T01:11:10.306 INFO:teuthology.orchestra.run.smithi150.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-22T01:11:10.313 INFO:teuthology.orchestra.run.smithi150.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-22T01:11:10.314 INFO:teuthology.orchestra.run.smithi150.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-22T01:11:10.315 INFO:teuthology.orchestra.run.smithi150.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-22T01:11:10.315 INFO:teuthology.orchestra.run.smithi150.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-22T01:11:10.316 INFO:teuthology.orchestra.run.smithi150.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-22T01:11:10.317 INFO:teuthology.orchestra.run.smithi150.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-22T01:11:10.317 INFO:teuthology.orchestra.run.smithi150.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-22T01:11:10.318 INFO:teuthology.orchestra.run.smithi150.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-22T01:11:10.319 INFO:teuthology.orchestra.run.smithi150.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-22T01:11:10.330 INFO:teuthology.orchestra.run.smithi150.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-22T01:11:10.337 INFO:teuthology.orchestra.run.smithi150.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-22T01:11:10.337 INFO:teuthology.orchestra.run.smithi150.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-22T01:11:10.338 INFO:teuthology.orchestra.run.smithi150.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-22T01:11:10.339 INFO:teuthology.orchestra.run.smithi150.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-22T01:11:10.340 INFO:teuthology.orchestra.run.smithi150.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-22T01:11:10.342 INFO:teuthology.orchestra.run.smithi150.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-22T01:11:10.343 INFO:teuthology.orchestra.run.smithi150.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-22T01:11:10.344 INFO:teuthology.orchestra.run.smithi150.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-22T01:11:10.348 INFO:teuthology.orchestra.run.smithi150.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-22T01:11:10.363 INFO:teuthology.orchestra.run.smithi150.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-22T01:11:10.363 INFO:teuthology.orchestra.run.smithi150.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-22T01:11:10.364 INFO:teuthology.orchestra.run.smithi150.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-22T01:11:10.366 INFO:teuthology.orchestra.run.smithi150.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-22T01:11:10.391 INFO:teuthology.orchestra.run.smithi150.stdout:Get:53 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-22T01:11:10.404 INFO:teuthology.orchestra.run.smithi150.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-22T01:11:10.411 INFO:teuthology.orchestra.run.smithi150.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-22T01:11:10.412 INFO:teuthology.orchestra.run.smithi150.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-22T01:11:10.415 INFO:teuthology.orchestra.run.smithi150.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-22T01:11:10.417 INFO:teuthology.orchestra.run.smithi150.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-22T01:11:10.419 INFO:teuthology.orchestra.run.smithi150.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-22T01:11:10.420 INFO:teuthology.orchestra.run.smithi150.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-22T01:11:10.421 INFO:teuthology.orchestra.run.smithi150.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-22T01:11:10.421 INFO:teuthology.orchestra.run.smithi150.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-22T01:11:10.422 INFO:teuthology.orchestra.run.smithi150.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-22T01:11:10.422 INFO:teuthology.orchestra.run.smithi150.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-22T01:11:10.425 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-373-g4e45e6b4-1jammy [3,241 kB] 2024-09-22T01:11:10.439 INFO:teuthology.orchestra.run.smithi150.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-22T01:11:10.441 INFO:teuthology.orchestra.run.smithi150.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-22T01:11:10.441 INFO:teuthology.orchestra.run.smithi150.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-22T01:11:10.456 INFO:teuthology.orchestra.run.smithi150.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-22T01:11:10.456 INFO:teuthology.orchestra.run.smithi150.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-22T01:11:10.456 INFO:teuthology.orchestra.run.smithi150.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-22T01:11:10.457 INFO:teuthology.orchestra.run.smithi150.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-22T01:11:10.457 INFO:teuthology.orchestra.run.smithi150.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-22T01:11:10.458 INFO:teuthology.orchestra.run.smithi150.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-22T01:11:10.473 INFO:teuthology.orchestra.run.smithi150.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-22T01:11:10.474 INFO:teuthology.orchestra.run.smithi150.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-22T01:11:10.477 INFO:teuthology.orchestra.run.smithi150.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-22T01:11:10.491 INFO:teuthology.orchestra.run.smithi150.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-22T01:11:10.492 INFO:teuthology.orchestra.run.smithi150.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-22T01:11:10.492 INFO:teuthology.orchestra.run.smithi150.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-22T01:11:10.495 INFO:teuthology.orchestra.run.smithi150.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-22T01:11:10.497 INFO:teuthology.orchestra.run.smithi150.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-22T01:11:10.498 INFO:teuthology.orchestra.run.smithi150.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-22T01:11:10.499 INFO:teuthology.orchestra.run.smithi150.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-22T01:11:10.508 INFO:teuthology.orchestra.run.smithi150.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-22T01:11:10.509 INFO:teuthology.orchestra.run.smithi150.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-22T01:11:10.512 INFO:teuthology.orchestra.run.smithi150.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-22T01:11:10.526 INFO:teuthology.orchestra.run.smithi150.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-22T01:11:10.526 INFO:teuthology.orchestra.run.smithi150.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-22T01:11:10.526 INFO:teuthology.orchestra.run.smithi150.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-22T01:11:10.528 INFO:teuthology.orchestra.run.smithi150.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-22T01:11:10.529 INFO:teuthology.orchestra.run.smithi150.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-22T01:11:10.535 INFO:teuthology.orchestra.run.smithi150.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-22T01:11:10.535 INFO:teuthology.orchestra.run.smithi150.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-22T01:11:10.538 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-373-g4e45e6b4-1jammy [719 kB] 2024-09-22T01:11:10.543 INFO:teuthology.orchestra.run.smithi150.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-22T01:11:10.545 INFO:teuthology.orchestra.run.smithi150.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-22T01:11:10.545 INFO:teuthology.orchestra.run.smithi150.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-22T01:11:10.570 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-373-g4e45e6b4-1jammy [356 kB] 2024-09-22T01:11:10.577 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-373-g4e45e6b4-1jammy [32.8 kB] 2024-09-22T01:11:10.578 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-373-g4e45e6b4-1jammy [180 kB] 2024-09-22T01:11:10.582 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-373-g4e45e6b4-1jammy [67.8 kB] 2024-09-22T01:11:10.583 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-373-g4e45e6b4-1jammy [333 kB] 2024-09-22T01:11:10.605 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-373-g4e45e6b4-1jammy [6,875 kB] 2024-09-22T01:11:10.634 INFO:teuthology.orchestra.run.smithi150.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-22T01:11:10.637 INFO:teuthology.orchestra.run.smithi150.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-22T01:11:10.647 INFO:teuthology.orchestra.run.smithi150.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-22T01:11:10.662 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-22T01:11:10.675 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-22T01:11:10.756 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-22T01:11:10.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-373-g4e45e6b4-1jammy [112 kB] 2024-09-22T01:11:10.859 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-373-g4e45e6b4-1jammy [472 kB] 2024-09-22T01:11:10.868 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-373-g4e45e6b4-1jammy [25.9 MB] 2024-09-22T01:11:10.986 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-22T01:11:10.997 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-22T01:11:11.026 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-22T01:11:11.035 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-22T01:11:11.036 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-22T01:11:11.038 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-22T01:11:11.055 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-22T01:11:11.060 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-22T01:11:11.065 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-22T01:11:11.068 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-22T01:11:11.149 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-22T01:11:11.153 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-22T01:11:11.158 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-22T01:11:11.161 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-22T01:11:11.162 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-22T01:11:11.163 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-22T01:11:11.165 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-22T01:11:11.165 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-22T01:11:11.166 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-22T01:11:11.231 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-22T01:11:11.249 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-22T01:11:11.250 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-22T01:11:11.250 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-22T01:11:11.256 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-22T01:11:11.257 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-22T01:11:11.314 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-22T01:11:11.314 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-22T01:11:11.316 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-22T01:11:11.317 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-22T01:11:11.394 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-22T01:11:11.397 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-22T01:11:11.399 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-22T01:11:11.400 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-22T01:11:11.410 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-22T01:11:11.476 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-22T01:11:11.477 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-22T01:11:11.478 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-22T01:11:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-22T01:11:11.485 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-22T01:11:11.508 INFO:teuthology.orchestra.run.smithi150.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-22T01:11:11.558 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-22T01:11:11.560 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-22T01:11:11.562 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-22T01:11:11.567 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-22T01:11:11.568 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-22T01:11:11.595 INFO:teuthology.orchestra.run.smithi150.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-22T01:11:11.599 INFO:teuthology.orchestra.run.smithi150.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-22T01:11:11.601 INFO:teuthology.orchestra.run.smithi150.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-22T01:11:11.639 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-22T01:11:11.642 INFO:teuthology.orchestra.run.smithi150.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-22T01:11:11.644 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-22T01:11:11.810 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-22T01:11:11.812 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-22T01:11:11.815 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-22T01:11:11.816 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-22T01:11:11.818 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-22T01:11:11.826 INFO:teuthology.orchestra.run.smithi150.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-22T01:11:11.866 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-22T01:11:11.947 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-22T01:11:11.947 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-22T01:11:11.949 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-373-g4e45e6b4-1jammy [5,145 kB] 2024-09-22T01:11:11.975 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-22T01:11:11.975 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-22T01:11:11.978 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-22T01:11:11.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-22T01:11:11.979 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-22T01:11:12.035 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-22T01:11:12.036 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-22T01:11:12.059 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-22T01:11:12.068 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-22T01:11:12.071 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-373-g4e45e6b4-1jammy [245 kB] 2024-09-22T01:11:12.073 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-22T01:11:12.075 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-373-g4e45e6b4-1jammy [124 kB] 2024-09-22T01:11:12.076 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-22T01:11:12.078 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-373-g4e45e6b4-1jammy [1,555 kB] 2024-09-22T01:11:12.078 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-22T01:11:12.128 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-373-g4e45e6b4-1jammy [6,142 kB] 2024-09-22T01:11:12.141 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-22T01:11:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-22T01:11:12.150 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-22T01:11:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-22T01:11:12.163 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-22T01:11:12.223 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-22T01:11:12.230 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-22T01:11:12.231 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-22T01:11:12.233 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-22T01:11:12.233 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-22T01:11:12.304 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-22T01:11:12.306 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-22T01:11:12.324 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-22T01:11:12.325 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-22T01:11:12.325 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-22T01:11:12.382 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-373-g4e45e6b4-1jammy [22.8 MB] 2024-09-22T01:11:12.386 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-22T01:11:12.388 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-22T01:11:12.522 INFO:teuthology.orchestra.run.smithi150.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-22T01:11:12.523 INFO:teuthology.orchestra.run.smithi150.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-22T01:11:12.540 INFO:teuthology.orchestra.run.smithi150.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-22T01:11:12.581 INFO:teuthology.orchestra.run.smithi150.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-22T01:11:12.590 INFO:teuthology.orchestra.run.smithi150.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-22T01:11:12.592 INFO:teuthology.orchestra.run.smithi150.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-22T01:11:12.652 INFO:teuthology.orchestra.run.smithi150.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-22T01:11:12.902 INFO:teuthology.orchestra.run.smithi150.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-22T01:11:12.902 INFO:teuthology.orchestra.run.smithi150.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-22T01:11:13.326 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-373-g4e45e6b4-1jammy [14.1 kB] 2024-09-22T01:11:13.326 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [877 kB] 2024-09-22T01:11:13.343 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-373-g4e45e6b4-1jammy [2,462 kB] 2024-09-22T01:11:13.391 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-373-g4e45e6b4-1jammy [794 kB] 2024-09-22T01:11:13.403 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-373-g4e45e6b4-1jammy [152 kB] 2024-09-22T01:11:13.405 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-373-g4e45e6b4-1jammy [2,347 kB] 2024-09-22T01:11:13.483 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-373-g4e45e6b4-1jammy [8,625 kB] 2024-09-22T01:11:13.812 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-373-g4e45e6b4-1jammy [14.2 kB] 2024-09-22T01:11:13.813 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-373-g4e45e6b4-1jammy [51.4 MB] 2024-09-22T01:11:14.629 INFO:teuthology.orchestra.run.smithi150.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-22T01:11:14.632 INFO:teuthology.orchestra.run.smithi150.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-22T01:11:14.633 INFO:teuthology.orchestra.run.smithi150.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-22T01:11:14.962 INFO:teuthology.orchestra.run.smithi150.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-22T01:11:15.482 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-373-g4e45e6b4-1jammy [130 kB] 2024-09-22T01:11:15.484 INFO:teuthology.orchestra.run.smithi096.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-373-g4e45e6b4-1jammy [39.0 kB] 2024-09-22T01:11:15.485 INFO:teuthology.orchestra.run.smithi096.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-373-g4e45e6b4-1jammy [13.6 MB] 2024-09-22T01:11:15.547 INFO:teuthology.orchestra.run.smithi150.stdout:Fetched 181 MB in 5s (36.3 MB/s) 2024-09-22T01:11:15.675 INFO:teuthology.orchestra.run.smithi096.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-373-g4e45e6b4-1jammy [91.4 kB] 2024-09-22T01:11:15.711 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-22T01:11:15.810 INFO:teuthology.orchestra.run.smithi150.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-22T01:11:15.815 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-22T01:11:15.900 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T01:11:16.220 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-22T01:11:16.245 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 181 MB in 6s (32.2 MB/s) 2024-09-22T01:11:16.247 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-22T01:11:16.287 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T01:11:16.421 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-22T01:11:16.522 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-22T01:11:16.528 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-22T01:11:16.603 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T01:11:16.615 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-22T01:11:16.641 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-22T01:11:16.682 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T01:11:16.948 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-22T01:11:16.975 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-22T01:11:17.015 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T01:11:17.052 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-22T01:11:17.078 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:17.118 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:17.343 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-22T01:11:17.369 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-22T01:11:17.409 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T01:11:17.530 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-22T01:11:17.556 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:17.596 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:17.788 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-22T01:11:17.814 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:17.854 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:17.874 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-22T01:11:17.900 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:17.933 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:18.249 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-22T01:11:18.260 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-22T01:11:18.275 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:18.286 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-22T01:11:18.316 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:18.327 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T01:11:18.669 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-22T01:11:18.672 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librados2. 2024-09-22T01:11:18.695 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-22T01:11:18.697 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:18.735 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:11:18.738 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:19.088 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-22T01:11:19.115 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-22T01:11:19.155 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T01:11:19.166 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librbd1. 2024-09-22T01:11:19.192 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:19.224 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:19.424 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librados2. 2024-09-22T01:11:19.440 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-librados2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:19.474 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:19.602 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs2. 2024-09-22T01:11:19.617 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:19.660 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:19.962 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rados. 2024-09-22T01:11:19.978 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:20.021 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.027 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librbd1. 2024-09-22T01:11:20.053 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-librbd1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:20.093 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.323 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-22T01:11:20.349 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:20.390 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.538 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2024-09-22T01:11:20.564 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-libcephfs2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:20.604 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.634 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cephfs. 2024-09-22T01:11:20.660 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:20.701 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.882 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2024-09-22T01:11:20.897 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-python3-rados_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:20.932 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:20.970 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-22T01:11:20.996 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:21.037 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:21.193 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-22T01:11:21.220 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:21.260 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:21.348 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-22T01:11:21.374 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-22T01:11:21.415 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T01:11:21.495 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2024-09-22T01:11:21.521 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-python3-cephfs_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:21.562 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:21.726 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-prettytable. 2024-09-22T01:11:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-22T01:11:21.793 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-22T01:11:21.881 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-22T01:11:21.908 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-python3-ceph-common_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:21.948 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:22.045 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rbd. 2024-09-22T01:11:22.071 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:22.259 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-22T01:11:22.285 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-22T01:11:22.325 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T01:11:22.328 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:22.595 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2024-09-22T01:11:22.610 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-22T01:11:22.645 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-22T01:11:22.648 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-22T01:11:22.674 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-22T01:11:22.715 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T01:11:22.889 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2024-09-22T01:11:22.905 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-python3-rbd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:22.948 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:22.967 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-22T01:11:22.993 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-22T01:11:23.034 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T01:11:23.292 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-22T01:11:23.319 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-22T01:11:23.337 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-22T01:11:23.358 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T01:11:23.363 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-22T01:11:23.403 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T01:11:23.698 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua5.1. 2024-09-22T01:11:23.724 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-22T01:11:23.756 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-22T01:11:23.852 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-22T01:11:23.868 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-22T01:11:23.911 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T01:11:24.051 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-any. 2024-09-22T01:11:24.077 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-22T01:11:24.134 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-22T01:11:24.214 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-22T01:11:24.229 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-22T01:11:24.272 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T01:11:24.378 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package zip. 2024-09-22T01:11:24.402 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-22T01:11:24.436 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking zip (3.0-12build2) ... 2024-09-22T01:11:24.575 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua5.1. 2024-09-22T01:11:24.601 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-22T01:11:24.642 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-22T01:11:24.690 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package unzip. 2024-09-22T01:11:24.704 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-22T01:11:24.748 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-22T01:11:24.953 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-any. 2024-09-22T01:11:24.980 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-22T01:11:25.019 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-22T01:11:25.144 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package luarocks. 2024-09-22T01:11:25.170 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-22T01:11:25.210 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-22T01:11:25.255 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package zip. 2024-09-22T01:11:25.271 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-22T01:11:25.314 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking zip (3.0-12build2) ... 2024-09-22T01:11:25.593 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package librgw2. 2024-09-22T01:11:25.619 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:25.642 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package unzip. 2024-09-22T01:11:25.659 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:25.668 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-22T01:11:25.708 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-22T01:11:26.070 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package luarocks. 2024-09-22T01:11:26.097 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-22T01:11:26.129 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-22T01:11:26.195 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rgw. 2024-09-22T01:11:26.210 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:26.245 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:26.505 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2024-09-22T01:11:26.532 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-librgw2_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:26.564 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-22T01:11:26.571 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:26.591 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-22T01:11:26.631 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T01:11:26.967 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libradosstriper1. 2024-09-22T01:11:26.993 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:27.034 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:27.232 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2024-09-22T01:11:27.259 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-python3-rgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:27.487 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-common. 2024-09-22T01:11:27.513 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:27.554 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:27.593 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-22T01:11:27.620 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-22T01:11:27.651 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T01:11:27.971 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2024-09-22T01:11:27.997 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-libradosstriper1_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:28.474 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2024-09-22T01:11:28.501 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-ceph-common_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:28.540 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:28.876 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-base. 2024-09-22T01:11:28.902 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:28.952 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:29.513 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-22T01:11:29.539 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-22T01:11:29.580 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-22T01:11:29.866 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cheroot. 2024-09-22T01:11:29.888 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2024-09-22T01:11:29.892 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-22T01:11:29.914 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-ceph-base_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:29.932 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T01:11:29.955 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:30.194 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-22T01:11:30.209 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-22T01:11:30.244 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-22T01:11:30.546 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-22T01:11:30.573 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-22T01:11:30.609 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-22T01:11:30.613 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-22T01:11:30.635 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-22T01:11:30.700 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-22T01:11:30.916 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-22T01:11:30.942 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-22T01:11:30.978 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cheroot. 2024-09-22T01:11:30.982 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-22T01:11:31.004 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-22T01:11:31.044 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T01:11:31.261 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempora. 2024-09-22T01:11:31.287 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-22T01:11:31.327 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-22T01:11:31.364 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-22T01:11:31.379 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-22T01:11:31.414 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-22T01:11:31.597 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-portend. 2024-09-22T01:11:31.612 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-22T01:11:31.655 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-22T01:11:31.725 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-22T01:11:31.751 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-22T01:11:31.792 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-22T01:11:31.957 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-22T01:11:31.984 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-22T01:11:32.024 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-22T01:11:32.094 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-22T01:11:32.121 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-22T01:11:32.161 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-22T01:11:32.438 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempora. 2024-09-22T01:11:32.465 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-22T01:11:32.505 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-22T01:11:32.542 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-22T01:11:32.568 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-22T01:11:32.619 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-22T01:11:32.783 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-portend. 2024-09-22T01:11:32.809 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-22T01:11:32.850 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-22T01:11:32.997 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-natsort. 2024-09-22T01:11:33.023 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-22T01:11:33.063 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-22T01:11:33.152 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-22T01:11:33.179 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-22T01:11:33.219 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-22T01:11:33.383 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-logutils. 2024-09-22T01:11:33.410 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-22T01:11:33.450 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-22T01:11:33.522 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-22T01:11:33.548 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-22T01:11:33.588 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-22T01:11:33.686 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-22T01:11:33.701 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-22T01:11:33.736 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-22T01:11:33.892 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-natsort. 2024-09-22T01:11:33.919 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-22T01:11:33.958 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-22T01:11:34.031 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-mako. 2024-09-22T01:11:34.057 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-22T01:11:34.097 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T01:11:34.295 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2024-09-22T01:11:34.321 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-22T01:11:34.361 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-22T01:11:34.434 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-22T01:11:34.460 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-22T01:11:34.500 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-22T01:11:34.622 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-22T01:11:34.649 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-22T01:11:34.680 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-22T01:11:34.803 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-22T01:11:34.830 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-22T01:11:34.869 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-22T01:11:35.130 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webob. 2024-09-22T01:11:35.146 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-22T01:11:35.180 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T01:11:35.183 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2024-09-22T01:11:35.210 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-22T01:11:35.250 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T01:11:35.458 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-waitress. 2024-09-22T01:11:35.485 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-22T01:11:35.528 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T01:11:35.578 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-22T01:11:35.605 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-22T01:11:35.644 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-22T01:11:35.836 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-tempita. 2024-09-22T01:11:35.863 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-22T01:11:35.903 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T01:11:35.947 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-22T01:11:35.974 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-22T01:11:36.014 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-22T01:11:36.189 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-paste. 2024-09-22T01:11:36.216 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-22T01:11:36.255 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T01:11:36.283 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2024-09-22T01:11:36.310 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-22T01:11:36.341 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T01:11:36.601 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-22T01:11:36.627 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-22T01:11:36.661 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2024-09-22T01:11:36.667 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T01:11:36.688 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-22T01:11:36.730 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T01:11:36.970 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-22T01:11:36.996 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-22T01:11:37.036 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-22T01:11:37.039 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2024-09-22T01:11:37.065 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-22T01:11:37.105 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T01:11:37.331 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-webtest. 2024-09-22T01:11:37.357 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-22T01:11:37.397 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-22T01:11:37.408 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2024-09-22T01:11:37.435 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-22T01:11:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T01:11:37.667 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pecan. 2024-09-22T01:11:37.694 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-22T01:11:37.734 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T01:11:37.786 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-22T01:11:37.802 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-22T01:11:37.869 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T01:11:38.064 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-certifi. 2024-09-22T01:11:38.090 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-22T01:11:38.130 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-22T01:11:38.189 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-22T01:11:38.216 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-22T01:11:38.255 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-22T01:11:38.592 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2024-09-22T01:11:38.619 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-22T01:11:38.659 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-22T01:11:38.716 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-idna. 2024-09-22T01:11:38.743 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-22T01:11:38.783 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T01:11:38.936 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2024-09-22T01:11:38.963 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-22T01:11:38.994 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T01:11:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-urllib3. 2024-09-22T01:11:39.104 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-22T01:11:39.144 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T01:11:39.316 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-certifi. 2024-09-22T01:11:39.343 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-22T01:11:39.382 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-22T01:11:39.464 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests. 2024-09-22T01:11:39.490 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-22T01:11:39.530 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T01:11:39.710 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-idna. 2024-09-22T01:11:39.737 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-22T01:11:39.777 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T01:11:39.842 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-22T01:11:39.868 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-22T01:11:39.908 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T01:11:40.088 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-urllib3. 2024-09-22T01:11:40.115 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-22T01:11:40.152 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-22T01:11:40.168 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:40.202 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:40.263 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T01:11:40.566 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-22T01:11:40.592 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:40.608 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests. 2024-09-22T01:11:40.632 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:40.634 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-22T01:11:40.674 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T01:11:40.902 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-22T01:11:40.917 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-22T01:11:40.951 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-22T01:11:40.985 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-22T01:11:41.012 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-22T01:11:41.052 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T01:11:41.238 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-openssl. 2024-09-22T01:11:41.264 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-22T01:11:41.304 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-22T01:11:41.346 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-22T01:11:41.362 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:41.446 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:41.657 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr. 2024-09-22T01:11:41.683 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:41.723 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:41.768 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-22T01:11:41.783 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:41.826 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:42.120 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-22T01:11:42.127 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mon. 2024-09-22T01:11:42.147 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-22T01:11:42.153 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:42.187 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-22T01:11:42.193 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:42.489 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2024-09-22T01:11:42.516 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-22T01:11:42.556 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-22T01:11:42.730 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-osd. 2024-09-22T01:11:42.757 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:42.876 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2024-09-22T01:11:42.941 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:42.941 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-ceph-mgr_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:42.942 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:43.312 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2024-09-22T01:11:43.339 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-ceph-mon_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:43.379 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:44.567 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph. 2024-09-22T01:11:44.582 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:44.625 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:44.757 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2024-09-22T01:11:44.784 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-ceph-osd_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:44.824 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:44.953 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-fuse. 2024-09-22T01:11:44.979 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:45.019 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:45.331 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mds. 2024-09-22T01:11:45.357 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:45.389 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:45.792 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package cephadm. 2024-09-22T01:11:45.818 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:45.874 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:45.877 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2024-09-22T01:11:45.892 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-ceph_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:45.927 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:46.163 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2024-09-22T01:11:46.186 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-22T01:11:46.189 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-ceph-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:46.213 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-22T01:11:46.229 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:46.252 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-22T01:11:46.563 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-22T01:11:46.579 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T01:11:46.599 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2024-09-22T01:11:46.621 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T01:11:46.626 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-ceph-mds_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:46.665 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:47.068 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2024-09-22T01:11:47.095 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-cephadm_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:47.135 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:47.429 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-22T01:11:47.456 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-22T01:11:47.487 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-22T01:11:47.627 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-babel. 2024-09-22T01:11:47.654 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T01:11:47.693 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T01:11:47.815 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-22T01:11:47.830 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T01:11:47.873 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T01:11:48.021 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-jinja2. 2024-09-22T01:11:48.048 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-22T01:11:48.088 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T01:11:48.466 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-22T01:11:48.493 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:48.532 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:48.846 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-22T01:11:48.873 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-22T01:11:48.904 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-22T01:11:48.936 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-babel. 2024-09-22T01:11:48.963 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-22T01:11:49.003 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T01:11:49.198 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-routes. 2024-09-22T01:11:49.225 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-22T01:11:49.265 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T01:11:49.331 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2024-09-22T01:11:49.357 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-22T01:11:49.397 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T01:11:49.626 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-22T01:11:49.653 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:49.693 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:49.767 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-22T01:11:49.794 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:49.833 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:50.170 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-22T01:11:50.197 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-22T01:11:50.236 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-22T01:11:50.731 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2024-09-22T01:11:50.758 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-22T01:11:50.797 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T01:11:50.957 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-22T01:11:50.984 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-22T01:11:51.031 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T01:11:51.134 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-22T01:11:51.161 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:51.200 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:51.577 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-joblib. 2024-09-22T01:11:51.604 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-22T01:11:51.651 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T01:11:52.038 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-22T01:11:52.065 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-22T01:11:52.105 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-22T01:11:52.374 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-sklearn. 2024-09-22T01:11:52.401 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-22T01:11:52.432 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T01:11:52.480 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-22T01:11:52.496 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-22T01:11:52.530 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T01:11:53.050 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2024-09-22T01:11:53.077 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-22T01:11:53.117 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T01:11:53.124 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-22T01:11:53.140 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:53.267 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:53.470 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-22T01:11:53.497 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-22T01:11:53.536 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-22T01:11:53.789 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2024-09-22T01:11:53.816 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-22T01:11:53.847 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T01:11:54.346 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-cachetools. 2024-09-22T01:11:54.373 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-22T01:11:54.412 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-22T01:11:54.631 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-22T01:11:54.658 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:54.690 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-22T01:11:54.706 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-22T01:11:54.740 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-22T01:11:54.783 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:55.018 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-22T01:11:55.045 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-22T01:11:55.085 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T01:11:55.396 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-rsa. 2024-09-22T01:11:55.424 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-22T01:11:55.463 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-22T01:11:55.708 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-google-auth. 2024-09-22T01:11:55.724 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-22T01:11:55.758 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-22T01:11:55.794 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cachetools. 2024-09-22T01:11:55.821 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-22T01:11:55.861 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-22T01:11:56.061 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-22T01:11:56.088 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-22T01:11:56.122 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-22T01:11:56.127 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T01:11:56.149 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-22T01:11:56.182 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-22T01:11:56.397 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-websocket. 2024-09-22T01:11:56.413 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-22T01:11:56.447 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-22T01:11:56.500 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-22T01:11:56.527 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-22T01:11:56.567 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T01:11:56.734 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-22T01:11:56.761 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-22T01:11:56.800 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T01:11:56.878 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2024-09-22T01:11:56.905 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-22T01:11:56.944 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-22T01:11:57.282 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-google-auth. 2024-09-22T01:11:57.309 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-22T01:11:57.348 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-22T01:11:57.481 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-22T01:11:57.508 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:57.547 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:57.676 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-22T01:11:57.704 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-22T01:11:57.743 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T01:11:57.833 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-22T01:11:57.860 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-22T01:11:57.900 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T01:11:58.054 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2024-09-22T01:11:58.081 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-22T01:11:58.120 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-22T01:11:58.178 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-22T01:11:58.205 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T01:11:58.244 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T01:11:58.441 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-22T01:11:58.468 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-22T01:11:58.505 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package jq. 2024-09-22T01:11:58.507 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T01:11:58.521 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T01:11:58.563 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-22T01:11:58.799 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package socat. 2024-09-22T01:11:58.815 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-22T01:11:58.858 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-22T01:11:59.177 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package xmlstarlet. 2024-09-22T01:11:59.187 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-22T01:11:59.193 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-22T01:11:59.215 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:11:59.227 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-22T01:11:59.253 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:59.556 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-test. 2024-09-22T01:11:59.572 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:11:59.606 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-22T01:11:59.614 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:11:59.635 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-22T01:11:59.673 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T01:11:59.985 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-22T01:12:00.012 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T01:12:00.042 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T01:12:00.262 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2024-09-22T01:12:00.277 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-22T01:12:00.312 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-22T01:12:00.556 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2024-09-22T01:12:00.583 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-22T01:12:00.622 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-22T01:12:01.159 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2024-09-22T01:12:01.186 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-22T01:12:01.225 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-22T01:12:01.572 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2024-09-22T01:12:01.599 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-ceph-test_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:01.638 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:01.794 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package ceph-volume. 2024-09-22T01:12:01.821 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:12:01.852 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:02.139 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-22T01:12:02.155 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:02.189 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:02.484 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-22T01:12:02.511 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-22T01:12:02.550 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T01:12:02.888 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-22T01:12:02.915 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-22T01:12:02.954 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-22T01:12:03.325 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package nvme-cli. 2024-09-22T01:12:03.352 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-22T01:12:03.391 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T01:12:04.017 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-volume. 2024-09-22T01:12:04.045 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-ceph-volume_19.1.1-373-g4e45e6b4-1jammy_all.deb ... 2024-09-22T01:12:04.080 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package pkg-config. 2024-09-22T01:12:04.083 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:04.108 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-22T01:12:04.147 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T01:12:04.408 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-22T01:12:04.423 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-22T01:12:04.429 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-22T01:12:04.457 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-libcephfs-dev_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:04.466 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-22T01:12:04.504 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:04.990 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-22T01:12:05.018 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-22T01:12:05.019 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-22T01:12:05.046 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-22T01:12:05.056 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T01:12:05.085 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-22T01:12:05.394 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-22T01:12:05.397 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pastescript. 2024-09-22T01:12:05.422 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-22T01:12:05.425 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-22T01:12:05.461 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-22T01:12:05.463 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-22T01:12:05.783 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pluggy. 2024-09-22T01:12:05.810 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-22T01:12:05.849 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-22T01:12:05.906 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2024-09-22T01:12:05.934 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-22T01:12:05.983 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T01:12:06.161 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-psutil. 2024-09-22T01:12:06.188 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-22T01:12:06.227 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-22T01:12:06.455 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pkg-config. 2024-09-22T01:12:06.483 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-22T01:12:06.538 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T01:12:06.539 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-py. 2024-09-22T01:12:06.566 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-22T01:12:06.605 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-22T01:12:06.791 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-22T01:12:06.807 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-22T01:12:06.850 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-22T01:12:06.925 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pygments. 2024-09-22T01:12:06.953 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-22T01:12:06.991 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T01:12:07.253 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-22T01:12:07.269 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-22T01:12:07.303 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-22T01:12:07.454 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-22T01:12:07.481 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-22T01:12:07.520 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-22T01:12:07.564 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2024-09-22T01:12:07.592 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-22T01:12:07.630 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-22T01:12:07.840 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-toml. 2024-09-22T01:12:07.867 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-22T01:12:07.906 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-22T01:12:07.951 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2024-09-22T01:12:07.979 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-22T01:12:08.017 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-22T01:12:08.176 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-pytest. 2024-09-22T01:12:08.203 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-22T01:12:08.233 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T01:12:08.329 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-psutil. 2024-09-22T01:12:08.357 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-22T01:12:08.395 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-22T01:12:08.587 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package python3-simplejson. 2024-09-22T01:12:08.615 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-22T01:12:08.653 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-22T01:12:08.732 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2024-09-22T01:12:08.759 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-22T01:12:08.806 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-22T01:12:08.931 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-22T01:12:08.948 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-22T01:12:08.989 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-22T01:12:09.127 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pygments. 2024-09-22T01:12:09.155 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-22T01:12:09.193 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T01:12:09.656 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-22T01:12:09.684 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-22T01:12:09.714 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-22T01:12:09.719 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package radosgw. 2024-09-22T01:12:09.746 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:09.785 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:09.975 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-toml. 2024-09-22T01:12:10.002 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-22T01:12:10.042 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-22T01:12:10.378 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2024-09-22T01:12:10.406 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-22T01:12:10.444 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T01:12:10.555 INFO:teuthology.orchestra.run.smithi150.stdout:Selecting previously unselected package rbd-fuse. 2024-09-22T01:12:10.583 INFO:teuthology.orchestra.run.smithi150.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:10.621 INFO:teuthology.orchestra.run.smithi150.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:10.807 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2024-09-22T01:12:10.834 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-22T01:12:10.873 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-22T01:12:10.925 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-22T01:12:11.159 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-22T01:12:11.176 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T01:12:11.187 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-22T01:12:11.226 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-22T01:12:11.302 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T01:12:11.558 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-22T01:12:11.907 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-22T01:12:12.171 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2024-09-22T01:12:12.199 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../116-radosgw_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:12.238 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:12.327 INFO:teuthology.orchestra.run.smithi150.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-22T01:12:12.395 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:12.895 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user cephadm....done 2024-09-22T01:12:13.033 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2024-09-22T01:12:13.051 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T01:12:13.060 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../117-rbd-fuse_19.1.1-373-g4e45e6b4-1jammy_amd64.deb ... 2024-09-22T01:12:13.099 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:13.328 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-22T01:12:13.404 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-22T01:12:13.579 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-22T01:12:13.680 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-22T01:12:13.688 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-22T01:12:13.805 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-22T01:12:13.931 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-22T01:12:14.068 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-22T01:12:14.174 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T01:12:14.300 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-22T01:12:14.406 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-22T01:12:14.602 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T01:12:14.795 INFO:teuthology.orchestra.run.smithi096.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-22T01:12:14.857 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:14.962 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-22T01:12:15.213 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-22T01:12:15.383 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2024-09-22T01:12:15.456 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-22T01:12:15.539 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-22T01:12:15.724 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-22T01:12:15.807 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-22T01:12:15.992 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-22T01:12:16.092 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-22T01:12:16.218 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-22T01:12:16.259 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-22T01:12:16.477 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-22T01:12:16.544 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:16.728 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-22T01:12:16.838 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-22T01:12:16.928 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T01:12:17.063 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T01:12:17.148 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-22T01:12:17.172 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T01:12:17.273 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-22T01:12:17.516 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-22T01:12:17.533 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-22T01:12:17.793 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-22T01:12:17.886 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-22T01:12:17.914 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-22T01:12:17.969 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T01:12:18.078 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-22T01:12:18.078 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-22T01:12:18.321 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-22T01:12:18.421 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T01:12:18.531 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-22T01:12:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-22T01:12:18.782 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T01:12:18.823 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-22T01:12:19.025 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-22T01:12:19.066 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:19.417 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-22T01:12:19.526 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up zip (3.0-12build2) ... 2024-09-22T01:12:19.543 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-22T01:12:19.668 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T01:12:19.669 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-22T01:12:19.795 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-22T01:12:20.063 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-22T01:12:20.345 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T01:12:20.449 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-22T01:12:20.486 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-22T01:12:20.541 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-22T01:12:20.588 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T01:12:20.667 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-22T01:12:20.714 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-22T01:12:20.815 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T01:12:21.027 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-22T01:12:21.099 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-22T01:12:21.153 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-22T01:12:21.420 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-22T01:12:21.492 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-22T01:12:21.680 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-22T01:12:21.752 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T01:12:21.964 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up zip (3.0-12build2) ... 2024-09-22T01:12:22.074 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-22T01:12:22.128 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T01:12:22.413 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T01:12:22.718 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-22T01:12:22.739 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-22T01:12:22.982 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-22T01:12:22.985 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-22T01:12:23.099 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:23.111 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-22T01:12:23.238 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-22T01:12:23.384 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T01:12:23.522 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-22T01:12:23.702 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T01:12:23.890 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-22T01:12:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-22T01:12:24.510 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-22T01:12:24.794 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-22T01:12:24.799 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T01:12:25.092 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:25.137 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-22T01:12:25.227 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-22T01:12:25.380 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-22T01:12:25.478 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T01:12:25.489 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:25.604 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-22T01:12:25.730 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T01:12:25.774 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-22T01:12:25.998 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-22T01:12:26.075 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-22T01:12:26.232 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-22T01:12:26.483 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:26.609 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-22T01:12:26.910 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-22T01:12:27.170 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-22T01:12:27.231 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-22T01:12:27.413 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-22T01:12:27.466 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:27.601 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-22T01:12:27.664 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-22T01:12:27.885 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-22T01:12:27.898 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-22T01:12:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-22T01:12:28.141 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T01:12:28.154 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-22T01:12:28.267 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-22T01:12:28.447 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-22T01:12:28.559 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T01:12:28.681 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-22T01:12:28.686 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T01:12:28.924 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:28.970 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T01:12:29.050 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-22T01:12:29.263 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-22T01:12:29.352 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-22T01:12:29.514 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T01:12:29.603 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-22T01:12:29.640 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-22T01:12:29.766 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T01:12:29.855 INFO:teuthology.orchestra.run.smithi150.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-22T01:12:29.871 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-22T01:12:30.084 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T01:12:30.155 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-22T01:12:30.210 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T01:12:30.398 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-22T01:12:30.595 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-22T01:12:30.666 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-22T01:12:30.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-22T01:12:30.838 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-22T01:12:30.964 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-22T01:12:31.068 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-22T01:12:31.194 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-22T01:12:31.215 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:31.341 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T01:12:31.479 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-22T01:12:31.659 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T01:12:31.771 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-22T01:12:31.960 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T01:12:32.006 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-22T01:12:32.107 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-22T01:12:32.225 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-22T01:12:32.236 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-22T01:12:32.314 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-22T01:12:32.337 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-22T01:12:32.543 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-22T01:12:32.596 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-22T01:12:32.669 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-22T01:12:32.873 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T01:12:33.054 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-22T01:12:33.217 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-22T01:12:33.305 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-22T01:12:33.423 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-22T01:12:33.594 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-22T01:12:33.682 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-22T01:12:33.808 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-22T01:12:33.879 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T01:12:34.134 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-22T01:12:34.223 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T01:12:34.349 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:34.435 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-22T01:12:34.475 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:34.601 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-22T01:12:34.712 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-22T01:12:34.821 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-22T01:12:35.079 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:35.080 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-22T01:12:35.190 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:35.316 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-22T01:12:35.374 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-22T01:12:35.601 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:35.719 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-22T01:12:35.727 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:35.828 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:36.027 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T01:12:36.027 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T01:12:36.112 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-22T01:12:36.380 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-22T01:12:36.521 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:36.639 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:36.691 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-22T01:12:36.783 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:36.817 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:36.917 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T01:12:36.943 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsqlite3-mod-ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:37.070 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-22T01:12:37.364 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:37.532 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:37.658 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-22T01:12:37.926 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.053 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.067 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.179 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.219 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.345 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.391 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T01:12:38.391 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-22T01:12:38.471 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.597 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:38.905 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:39.015 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:39.090 INFO:teuthology.orchestra.run.smithi150.stdout:Adding group ceph....done 2024-09-22T01:12:39.457 INFO:teuthology.orchestra.run.smithi150.stdout:Adding system user ceph....done 2024-09-22T01:12:39.500 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:39.626 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-22T01:12:39.627 INFO:teuthology.orchestra.run.smithi150.stdout:Setting system user ceph properties....done 2024-09-22T01:12:39.640 INFO:teuthology.orchestra.run.smithi150.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-22T01:12:39.778 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-22T01:12:40.123 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-22T01:12:40.573 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:40.690 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:40.822 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:40.923 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:41.032 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:41.152 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T01:12:41.152 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T01:12:41.158 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:41.284 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:41.601 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:41.827 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2024-09-22T01:12:41.948 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-22T01:12:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2024-09-22T01:12:42.402 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:42.406 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2024-09-22T01:12:42.425 INFO:teuthology.orchestra.run.smithi096.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-22T01:12:42.564 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-22T01:12:42.639 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T01:12:42.639 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T01:12:42.911 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-22T01:12:43.113 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:43.329 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T01:12:43.329 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T01:12:43.386 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:43.512 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:43.788 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:44.036 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T01:12:44.036 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T01:12:44.074 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T01:12:44.074 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-22T01:12:44.507 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:44.513 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:44.633 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:44.784 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:44.854 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-22T01:12:44.996 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T01:12:44.996 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T01:12:45.307 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:45.527 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T01:12:45.528 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-22T01:12:45.773 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:45.933 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:45.967 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:46.059 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:46.187 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T01:12:46.187 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-22T01:12:46.210 INFO:teuthology.orchestra.run.smithi150.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:46.579 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-22T01:12:46.660 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-22T01:12:46.832 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-22T01:12:46.926 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T01:12:46.926 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-22T01:12:47.582 INFO:teuthology.orchestra.run.smithi150.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-22T01:12:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:47.760 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:47.911 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:48.125 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T01:12:48.126 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-22T01:12:48.215 INFO:teuthology.orchestra.run.smithi150.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-22T01:12:48.220 DEBUG:teuthology.parallel:result is None 2024-09-22T01:12:48.613 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:48.789 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:48.932 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:49.083 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-volume (19.1.1-373-g4e45e6b4-1jammy) ... 2024-09-22T01:12:49.437 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-22T01:12:49.623 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-22T01:12:49.859 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-22T01:12:51.082 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-22T01:12:51.744 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-22T01:12:51.748 DEBUG:teuthology.parallel:result is None 2024-09-22T01:12:51.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T01:12:51.893 DEBUG:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2024-09-22T01:12:51.911 INFO:teuthology.orchestra.run.smithi096.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:12:51.911 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:12:51.911 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-22T01:12:51.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-22T01:12:52.051 DEBUG:teuthology.orchestra.run.smithi150:> dpkg-query -W -f '${Version}' ceph 2024-09-22T01:12:52.070 INFO:teuthology.orchestra.run.smithi150.stdout:19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:12:52.071 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373-g4e45e6b4-1jammy 2024-09-22T01:12:52.071 INFO:teuthology.task.install:The correct ceph version 19.1.1-373-g4e45e6b4-1jammy is installed. 2024-09-22T01:12:52.073 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-22T01:12:52.073 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:52.073 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T01:12:52.088 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:52.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-22T01:12:52.126 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-22T01:12:52.126 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:52.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T01:12:52.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T01:12:52.200 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:52.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-22T01:12:52.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-22T01:12:52.267 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-22T01:12:52.267 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:52.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T01:12:52.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T01:12:52.335 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:52.335 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-22T01:12:52.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-22T01:12:52.403 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-22T01:12:52.403 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:52.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T01:12:52.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T01:12:52.471 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:52.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-22T01:12:52.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-22T01:12:52.535 INFO:teuthology.run_tasks:Running task ceph... 2024-09-22T01:12:52.605 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-22T01:12:52.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-09-22T01:12:52.607 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-09-22T01:12:52.619 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-22T01:12:52.620 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T01:12:52.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-22T01:12:52.673 INFO:tasks.ceph:Creating extra log directories... 2024-09-22T01:12:52.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T01:12:52.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-22T01:12:52.739 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-22T01:12:52.739 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-22T01:12:52.739 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914605', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKV5b2WKvQmx8ptEvimAe8dS8jSsPqoNYlaLLOzFyqNe9zJxjKyLFOfbJHqwLkaRbeHcg/kfkqmiE9sh3JJQtts=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDOeOlqVTw6+W0xSuoRg+T2rxt2lEo4rz8gHxYTeTJHFle2XCKzhMNq9Z8gTNzYotitNw2QP/VJh7CZ3VmFjOYg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-22T01:12:52.740 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T01:12:52.773 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-22T01:12:52.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T01:12:52.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-09-22T01:12:52.834 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:52.834 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T01:12:52.880 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T01:12:52.880 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-22T01:12:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T01:12:52.928 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:52.928 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-09-22T01:12:52.928 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:52.928 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-22 01:07:28.613371058 +0000 2024-09-22T01:12:52.929 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-22 01:07:28.357373346 +0000 2024-09-22T01:12:52.929 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-22 01:07:28.357373346 +0000 2024-09-22T01:12:52.929 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-22T01:12:52.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T01:12:52.982 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-22T01:12:52.983 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-22T01:12:52.983 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00044532 s, 1.1 MB/s 2024-09-22T01:12:52.984 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T01:12:53.033 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-22T01:12:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-22 01:07:29.121366520 +0000 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-22 01:07:28.869368772 +0000 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-22 01:07:28.869368772 +0000 2024-09-22T01:12:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-22T01:12:53.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T01:12:53.135 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-22T01:12:53.135 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-22T01:12:53.135 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000345932 s, 1.5 MB/s 2024-09-22T01:12:53.136 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T01:12:53.185 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-22T01:12:53.232 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-22 01:07:29.369364306 +0000 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-22 01:07:29.365364342 +0000 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-22 01:07:29.365364342 +0000 2024-09-22T01:12:53.233 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-22T01:12:53.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T01:12:53.287 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-22T01:12:53.287 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-22T01:12:53.287 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000415837 s, 1.2 MB/s 2024-09-22T01:12:53.288 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T01:12:53.337 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-22T01:12:53.384 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T01:12:53.384 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.384 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-09-22T01:12:53.384 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.385 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-22 01:07:30.137357446 +0000 2024-09-22T01:12:53.385 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-22 01:07:29.885359697 +0000 2024-09-22T01:12:53.385 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-22 01:07:29.885359697 +0000 2024-09-22T01:12:53.385 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-22T01:12:53.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T01:12:53.438 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-22T01:12:53.438 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-22T01:12:53.438 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000379967 s, 1.3 MB/s 2024-09-22T01:12:53.439 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T01:12:53.490 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:53.490 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:53.490 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-22T01:12:53.495 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-22T01:12:53.496 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-22 01:07:30.030262690 +0000 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-22 01:07:29.774254280 +0000 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-22 01:07:29.774254280 +0000 2024-09-22T01:12:53.541 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-22T01:12:53.542 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-22T01:12:53.595 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-22T01:12:53.595 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-22T01:12:53.595 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000448527 s, 1.1 MB/s 2024-09-22T01:12:53.596 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-22T01:12:53.646 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-22T01:12:53.694 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-22T01:12:53.694 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.694 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-22T01:12:53.695 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.695 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-22 01:07:30.498277229 +0000 2024-09-22T01:12:53.695 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-22 01:07:30.274270271 +0000 2024-09-22T01:12:53.695 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-22 01:07:30.274270271 +0000 2024-09-22T01:12:53.695 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-22T01:12:53.695 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-22T01:12:53.746 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-22T01:12:53.746 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-22T01:12:53.747 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000330831 s, 1.5 MB/s 2024-09-22T01:12:53.748 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-22T01:12:53.794 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-22T01:12:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-22 01:07:30.754285183 +0000 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-22 01:07:30.750285059 +0000 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-22 01:07:30.750285059 +0000 2024-09-22T01:12:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-22T01:12:53.841 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-22T01:12:53.895 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-22T01:12:53.895 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-22T01:12:53.895 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000419016 s, 1.2 MB/s 2024-09-22T01:12:53.896 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-22T01:12:53.946 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-22T01:12:53.992 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-22 01:07:31.486307925 +0000 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-22 01:07:31.250300593 +0000 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-22 01:07:31.250300593 +0000 2024-09-22T01:12:53.993 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2024-09-22T01:12:53.993 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-22T01:12:54.046 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-22T01:12:54.047 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-22T01:12:54.047 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000273165 s, 1.9 MB/s 2024-09-22T01:12:54.048 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-22T01:12:54.098 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:54.098 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-22T01:12:54.098 INFO:tasks.ceph:Generating config... 2024-09-22T01:12:54.099 INFO:tasks.ceph:[client] rbd cache = True 2024-09-22T01:12:54.099 INFO:tasks.ceph:[client] rbd cache policy = writeback 2024-09-22T01:12:54.099 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-22T01:12:54.099 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-22T01:12:54.099 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-22T01:12:54.099 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-22T01:12:54.099 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-22T01:12:54.100 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-22T01:12:54.100 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-22T01:12:54.100 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] bluestore allocator = stupid 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-22T01:12:54.100 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-22T01:12:54.101 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-22T01:12:54.101 INFO:tasks.ceph:Setting up mon.a... 2024-09-22T01:12:54.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-22T01:12:54.135 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.keyring 2024-09-22T01:12:54.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-22T01:12:54.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T01:12:54.287 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.96'), ('mon.c', '[v2:172.21.15.96:3301,v1:172.21.15.96:6790]'), ('mon.b', '172.21.15.150')] 2024-09-22T01:12:54.287 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.150', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writeback'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-22T01:12:54.288 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:54.288 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T01:12:54.332 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.96 --addv c '[v2:172.21.15.96:3301,v1:172.21.15.96:6790]' --add b 172.21.15.150 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: generated fsid e212681a-0600-4f32-ab96-8ca46d37e9a4 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 0 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:fsid e212681a-0600-4f32-ab96-8ca46d37e9a4 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:last_changed 2024-09-22T01:12:54.417679+0000 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-22T01:12:54.417679+0000 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:min_mon_release 17 (quincy) 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:election_strategy: 1 2024-09-22T01:12:54.417 INFO:teuthology.orchestra.run.smithi096.stdout:0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-22T01:12:54.418 INFO:teuthology.orchestra.run.smithi096.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-09-22T01:12:54.418 INFO:teuthology.orchestra.run.smithi096.stdout:2: [v2:172.21.15.96:3301/0,v1:172.21.15.96:6790/0] mon.c 2024-09-22T01:12:54.418 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-22T01:12:54.419 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-22T01:12:54.463 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e212681a-0600-4f32-ab96-8ca46d37e9a4... 2024-09-22T01:12:54.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T01:12:54.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-22T01:12:54.534 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-09-22T01:12:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-09-22T01:12:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T01:12:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-09-22T01:12:54.534 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-09-22T01:12:54.535 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-09-22T01:12:54.536 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.150" 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon client directed command retry = 5 2024-09-22T01:12:54.537 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on pool no app = False 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject socket failures = 5000 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = e212681a-0600-4f32-ab96-8ca46d37e9a4 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-09-22T01:12:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug trim objects = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.539 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bdev async discard = True 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bdev enable discard = True 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore allocator = stupid 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 1/20 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 1/20 2024-09-22T01:12:54.540 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 4/10 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: osd blocked scrub grace period = 3600 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: osd max markdown count = 1000 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: osd shutdown pgref assert = True 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-22T01:12:54.541 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-09-22T01:12:54.542 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.543 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: rbd cache = True 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout: rbd cache policy = writeback 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-09-22T01:12:54.544 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-09-22T01:12:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-09-22T01:12:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-22T01:12:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.150" 2024-09-22T01:12:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on pool no app = False 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 5000 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = e212681a-0600-4f32-ab96-8ca46d37e9a4 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-09-22T01:12:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = stupid 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-09-22T01:12:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: osd blocked scrub grace period = 3600 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: osd max markdown count = 1000 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: osd shutdown pgref assert = True 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-09-22T01:12:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-09-22T01:12:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-22T01:12:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: rbd cache = True 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: rbd cache policy = writeback 2024-09-22T01:12:54.555 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-09-22T01:12:54.556 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2024-09-22T01:12:54.556 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2024-09-22T01:12:54.557 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-22T01:12:54.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-22T01:12:54.649 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-22T01:12:54.649 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:54.649 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-22T01:12:54.654 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:54.654 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-22T01:12:54.700 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:12:54.700 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:54.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T01:12:54.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T01:12:54.765 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:12:54.765 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:12:54.813 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:12:54.813 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:54.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-22T01:12:54.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-22T01:12:54.836 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:12:54.836 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:12:54.885 INFO:tasks.ceph:Setting up mon nodes... 2024-09-22T01:12:54.885 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-22T01:12:54.885 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-22T01:12:54.928 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-22T01:12:54.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T01:12:54.974 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-22T01:12:54.988 INFO:tasks.ceph:Setting up mds nodes... 2024-09-22T01:12:54.988 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-22T01:12:54.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-22T01:12:55.024 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-22T01:12:55.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-22T01:12:55.071 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-22T01:12:55.083 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-22T01:12:55.083 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-22T01:12:55.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-22T01:12:55.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:55.095 INFO:tasks.ceph:role: osd.0 2024-09-22T01:12:55.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:12:55.096 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T01:12:55.160 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:12:55.160 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:12:55.160 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:12:55.161 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:12:55.655 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-22T01:12:55.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-22T01:12:55.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-22T01:12:55.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-22T01:12:55.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:12:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:12:55.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-22T01:12:55.868 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:55.868 INFO:tasks.ceph:role: osd.1 2024-09-22T01:12:55.868 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:12:55.868 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T01:12:55.931 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:12:55.931 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:12:55.931 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:12:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:12:55.932 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:12:55.932 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:12:55.932 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:12:55.933 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:12:55.934 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:12:55.934 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:12:56.381 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-22T01:12:56.384 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-22T01:12:56.384 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-22T01:12:56.468 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-22T01:12:56.522 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:12:56.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:12:56.523 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-22T01:12:56.576 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:56.576 INFO:tasks.ceph:role: osd.2 2024-09-22T01:12:56.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:12:56.576 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:12:56.641 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:12:57.102 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-22T01:12:57.104 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-22T01:12:57.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-22T01:12:57.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-22T01:12:57.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:12:57.244 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:12:57.244 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-22T01:12:57.301 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-22T01:12:57.301 INFO:tasks.ceph:role: osd.3 2024-09-22T01:12:57.301 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:12:57.301 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:12:57.365 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:12:57.366 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:12:57.862 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-22T01:12:57.864 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-22T01:12:57.864 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-22T01:12:58.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-22T01:12:58.041 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:12:58.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:12:58.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:12:58.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:12:58.160+0000 7f51017d37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-22T01:12:58.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:12:58.160+0000 7f51017d37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-22T01:12:58.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:12:58.160+0000 7f51017d37c0 -1 bdev(0x55a27aa21800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-22T01:12:58.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:12:58.160+0000 7f51017d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-22T01:13:01.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-22T01:13:01.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:01.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:01.644+0000 7f80afaa87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-22T01:13:01.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:01.644+0000 7f80afaa87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-22T01:13:01.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:01.644+0000 7f80afaa87c0 -1 bdev(0x5591f484f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:01.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:01.644+0000 7f80afaa87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-22T01:13:04.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-22T01:13:04.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:04.912+0000 7efd348667c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-22T01:13:04.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:04.912+0000 7efd348667c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-22T01:13:04.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:04.912+0000 7efd348667c0 -1 bdev(0x55d8ccee3800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:04.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:04.912+0000 7efd348667c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-22T01:13:08.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-22T01:13:08.091 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:08.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:08.168+0000 7f0f2f20b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-22T01:13:08.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:08.168+0000 7f0f2f20b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-22T01:13:08.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:08.168+0000 7f0f2f20b7c0 -1 bdev(0x5647b01f9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:08.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:13:08.168+0000 7f0f2f20b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-22T01:13:11.307 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-22T01:13:11.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-22T01:13:11.375 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T01:13:11.375 INFO:tasks.ceph:role: osd.4 2024-09-22T01:13:11.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:13:11.375 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-22T01:13:11.436 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:13:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:13:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:13:11.436 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:13:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:13:11.893 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-22T01:13:11.895 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-22T01:13:11.896 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-22T01:13:11.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-22T01:13:12.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:13:12.001 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:13:12.001 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-22T01:13:12.055 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T01:13:12.055 INFO:tasks.ceph:role: osd.5 2024-09-22T01:13:12.055 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:13:12.055 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:13:12.116 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:13:12.578 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-22T01:13:12.580 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-22T01:13:12.580 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-22T01:13:12.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-22T01:13:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:13:12.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:13:12.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-22T01:13:12.804 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T01:13:12.804 INFO:tasks.ceph:role: osd.6 2024-09-22T01:13:12.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:13:12.804 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:13:12.868 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:13:13.334 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-22T01:13:13.336 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-22T01:13:13.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-22T01:13:13.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-22T01:13:13.508 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:13:13.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:13:13.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-22T01:13:13.565 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-22T01:13:13.565 INFO:tasks.ceph:role: osd.7 2024-09-22T01:13:13.565 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:13:13.565 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-22T01:13:13.627 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-22T01:13:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-22T01:13:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-22T01:13:13.627 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-22T01:13:13.628 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-22T01:13:14.054 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-09-22T01:13:14.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-09-22T01:13:14.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-22T01:13:14.149 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-22T01:13:14.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:13:14.159 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: /sbin/restorecon: command not found 2024-09-22T01:13:14.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:14.271 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:14.270+0000 7f1a5da567c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-22T01:13:14.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:14.274+0000 7f1a5da567c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-22T01:13:14.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:14.274+0000 7f1a5da567c0 -1 bdev(0x559745a87800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:14.273 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:14.274+0000 7f1a5da567c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-22T01:13:17.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-22T01:13:17.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:17.527 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:17.526+0000 7f04627847c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-22T01:13:17.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:17.526+0000 7f04627847c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-22T01:13:17.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:17.526+0000 7f04627847c0 -1 bdev(0x56437a563800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:17.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:17.530+0000 7f04627847c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-22T01:13:20.660 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-22T01:13:20.717 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:20.803 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:20.802+0000 7f773cf627c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-22T01:13:20.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:20.806+0000 7f773cf627c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-22T01:13:20.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:20.806+0000 7f773cf627c0 -1 bdev(0x55b691c17800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:20.804 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:20.806+0000 7f773cf627c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-22T01:13:23.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-22T01:13:24.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:24.122+0000 7f6548a117c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-22T01:13:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:24.122+0000 7f6548a117c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-22T01:13:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:24.122+0000 7f6548a117c0 -1 bdev(0x55f95ddfd800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-22T01:13:24.123 INFO:teuthology.orchestra.run.smithi150.stderr:2024-09-22T01:13:24.122+0000 7f6548a117c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-22T01:13:27.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-22T01:13:27.344 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-22T01:13:27.344 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-22T01:13:27.359 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-22T01:13:27.415 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.416 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-22T01:13:27.471 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-22T01:13:27.528 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.528 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-22T01:13:27.583 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-22T01:13:27.596 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.597 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-22T01:13:27.652 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.653 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-22T01:13:27.708 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-22T01:13:27.763 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-22T01:13:27.819 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:27.820 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-22T01:13:27.825 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:27.825 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-22T01:13:27.868 INFO:tasks.ceph:Adding keys to all mons... 2024-09-22T01:13:27.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T01:13:27.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-22T01:13:27.914 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.y] 2024-09-22T01:13:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAWb+9mWUhjNxAAllT3R48hU7/3l5sYJEmOdg== 2024-09-22T01:13:27.914 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-09-22T01:13:27.914 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAab+9m2WGYCRAAVo35k0zElQMA2un73DmC2g== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAdb+9m/nOGJhAAOeuV6f+TzsJ02UjTx4Cq4g== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAgb+9mOKGLNhAA/GcL0V66tMUDJYX1i1hjew== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAkb+9mcEIgChAAvO8CZZDHLqcmURcTE9qJOA== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAWb+9m7e8wOhAA1lGmKYcKLy4Ja35uxP+VFg== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAqb+9mkuNPEBAARK2SufCbgZQi1JOBetM/cA== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAtb+9m25KOHxAAl9rjIxNYYvbZ1oX7d4h4/g== 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.6] 2024-09-22T01:13:27.915 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAwb+9mI9EKMBAAkfupOjdnnZjOhKiuUq8Y2Q== 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.7] 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQA0b+9mSwx0BxAAaSginImz0zAVNCJLsYMTdg== 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAXb+9mJKV/ARAAf66JseWAlWJrX4mm8W6Jdg== 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-09-22T01:13:27.916 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQAXb+9mcEBnBBAAwRw6G5n9r/KvONTSqPgRJQ== 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAWb+9mWUhjNxAAllT3R48hU7/3l5sYJEmOdg== 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAab+9m2WGYCRAAVo35k0zElQMA2un73DmC2g== 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAdb+9m/nOGJhAAOeuV6f+TzsJ02UjTx4Cq4g== 2024-09-22T01:13:27.923 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAgb+9mOKGLNhAA/GcL0V66tMUDJYX1i1hjew== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAkb+9mcEIgChAAvO8CZZDHLqcmURcTE9qJOA== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAWb+9m7e8wOhAA1lGmKYcKLy4Ja35uxP+VFg== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAqb+9mkuNPEBAARK2SufCbgZQi1JOBetM/cA== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAtb+9m25KOHxAAl9rjIxNYYvbZ1oX7d4h4/g== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAwb+9mI9EKMBAAkfupOjdnnZjOhKiuUq8Y2Q== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQA0b+9mSwx0BxAAaSginImz0zAVNCJLsYMTdg== 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-09-22T01:13:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAXb+9mJKV/ARAAf66JseWAlWJrX4mm8W6Jdg== 2024-09-22T01:13:27.925 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-22T01:13:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAXb+9mcEBnBBAAwRw6G5n9r/KvONTSqPgRJQ== 2024-09-22T01:13:27.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:13:27.961 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:13:28.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.101 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.185 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:13:28.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-22T01:13:28.432 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.684 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-22T01:13:28.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:13:28.784 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:13:28.863 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:13:28.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-22T01:13:28.913 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-22T01:13:28.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-22T01:13:28.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:13:29.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-22T01:13:29.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-22T01:13:29.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:13:29.220 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-22T01:13:29.231 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-22T01:13:29.245 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-22T01:13:29.341 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-22T01:13:29.352 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:29.354 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-22T01:13:29.400 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-22T01:13:29.400 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-22T01:13:29.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-22T01:13:29.403 INFO:tasks.ceph.mon.a:Started 2024-09-22T01:13:29.403 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-22T01:13:29.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-22T01:13:29.405 INFO:tasks.ceph.mon.c:Started 2024-09-22T01:13:29.405 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-22T01:13:29.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-22T01:13:29.446 INFO:tasks.ceph.mon.b:Started 2024-09-22T01:13:29.446 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-22T01:13:29.446 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-22T01:13:29.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-22T01:13:29.449 INFO:tasks.ceph.mgr.y:Started 2024-09-22T01:13:29.449 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-22T01:13:29.449 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-22T01:13:29.451 INFO:tasks.ceph.mgr.x:Started 2024-09-22T01:13:29.451 INFO:tasks.ceph:Setting crush tunables to default 2024-09-22T01:13:29.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd crush tunables default 2024-09-22T01:13:34.947 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-09-22T01:13:34.966 INFO:tasks.ceph:check_enable_crimson: False 2024-09-22T01:13:34.966 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-22T01:13:34.966 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:34.966 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-22T01:13:34.978 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:34.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-22T01:13:35.029 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:35.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-22T01:13:35.049 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:35.050+0000 7f6247d2b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T01:13:35.088 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-22T01:13:35.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-22T01:13:35.145 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:35.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-22T01:13:35.158 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:35.158 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-22T01:13:35.215 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:35.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-22T01:13:35.267 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-22T01:13:35.267 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-22T01:13:35.275 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:35.272+0000 7ffb9e30c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-22T01:13:35.322 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new a0e12724-c6a8-43ce-aad2-d3043c2f23d9 0 2024-09-22T01:13:35.543 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:35.542+0000 7f6247d2b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T01:13:35.587 INFO:teuthology.orchestra.run.smithi150.stdout:0 2024-09-22T01:13:35.604 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new bd73805d-596e-40f0-ba54-69ca84175c51 1 2024-09-22T01:13:35.746 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:35.746+0000 7f6247d2b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T01:13:35.786 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:35.784+0000 7ffb9e30c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-22T01:13:35.808 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-09-22T01:13:35.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6 2 2024-09-22T01:13:36.018 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.016+0000 7ffb9e30c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-22T01:13:36.028 INFO:teuthology.orchestra.run.smithi150.stdout:2 2024-09-22T01:13:36.044 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new eed7a22f-aa64-4dad-9151-8fc05271275e 3 2024-09-22T01:13:36.249 INFO:teuthology.orchestra.run.smithi150.stdout:3 2024-09-22T01:13:36.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 14ed2997-6237-4ca6-a17b-0c0ae87f0645 4 2024-09-22T01:13:36.308 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.310+0000 7f6247d2b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T01:13:36.363 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.362+0000 7f6247d2b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T01:13:36.419 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.418+0000 7f6247d2b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T01:13:36.468 INFO:teuthology.orchestra.run.smithi150.stdout:4 2024-09-22T01:13:36.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 6ecd111f-9cc2-45a6-92f3-67f73bacfdad 5 2024-09-22T01:13:36.499 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.498+0000 7f6247d2b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T01:13:36.552 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.554+0000 7f6247d2b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T01:13:36.595 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.592+0000 7ffb9e30c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-22T01:13:36.633 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.634+0000 7f6247d2b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T01:13:36.650 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.648+0000 7ffb9e30c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-22T01:13:36.687 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.686+0000 7f6247d2b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T01:13:36.696 INFO:teuthology.orchestra.run.smithi150.stdout:5 2024-09-22T01:13:36.701 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.700+0000 7ffb9e30c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-22T01:13:36.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 5032e618-100c-4318-9825-3e90a3c4b25e 6 2024-09-22T01:13:36.778 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.776+0000 7ffb9e30c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-22T01:13:36.830 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.828+0000 7ffb9e30c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-22T01:13:36.845 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.846+0000 7f6247d2b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T01:13:36.908 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.908+0000 7ffb9e30c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-22T01:13:36.908 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:36.910+0000 7f6247d2b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T01:13:36.913 INFO:teuthology.orchestra.run.smithi150.stdout:6 2024-09-22T01:13:36.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 020cdc04-e2da-4af5-b6d9-674e88425186 7 2024-09-22T01:13:36.961 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:36.960+0000 7ffb9e30c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-22T01:13:37.111 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:37.108+0000 7ffb9e30c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-22T01:13:37.130 INFO:teuthology.orchestra.run.smithi150.stdout:7 2024-09-22T01:13:37.146 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-22T01:13:37.147 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-22T01:13:37.148 INFO:tasks.ceph.osd.0:Started 2024-09-22T01:13:37.149 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-22T01:13:37.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-22T01:13:37.150 INFO:tasks.ceph.osd.1:Started 2024-09-22T01:13:37.151 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-22T01:13:37.151 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-22T01:13:37.152 INFO:tasks.ceph.osd.2:Started 2024-09-22T01:13:37.153 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-22T01:13:37.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-22T01:13:37.154 INFO:tasks.ceph.osd.3:Started 2024-09-22T01:13:37.154 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-22T01:13:37.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-22T01:13:37.157 INFO:tasks.ceph.osd.4:Started 2024-09-22T01:13:37.157 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-22T01:13:37.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-22T01:13:37.158 INFO:tasks.ceph.osd.5:Started 2024-09-22T01:13:37.159 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-22T01:13:37.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-22T01:13:37.160 INFO:tasks.ceph.osd.6:Started 2024-09-22T01:13:37.161 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-22T01:13:37.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-22T01:13:37.162 INFO:tasks.ceph.osd.7:Started 2024-09-22T01:13:37.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:37.179 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:37.176+0000 7ffb9e30c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-22T01:13:37.358 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:13:37.359 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":10,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:37.127631+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:37.374 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-22T01:13:37.374 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-22T01:13:37.552 INFO:tasks.ceph.mgr.x.smithi150.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-22T01:13:37.552 INFO:tasks.ceph.mgr.x.smithi150.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-22T01:13:37.552 INFO:tasks.ceph.mgr.x.smithi150.stderr: from numpy import show_config as show_numpy_config 2024-09-22T01:13:37.634 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:37.634+0000 7f6247d2b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T01:13:37.819 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:37.818+0000 7f6247d2b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:13:37.823 INFO:tasks.ceph.mgr.y.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-22T01:13:37.823 INFO:tasks.ceph.mgr.y.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-22T01:13:37.823 INFO:tasks.ceph.mgr.y.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-22T01:13:37.904 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:37.904+0000 7ffb9e30c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-22T01:13:37.981 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:37.982+0000 7f6247d2b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T01:13:38.050 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:38.050+0000 7f6247d2b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T01:13:38.073 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:38.072+0000 7ffb9e30c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:13:38.179 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:38.184 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:38.186+0000 7f6247d2b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:13:38.228 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:38.228+0000 7ffb9e30c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-22T01:13:38.294 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:38.292+0000 7ffb9e30c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-22T01:13:38.357 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-22T01:13:38.357 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":10,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:37.127631+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:38.372 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-22T01:13:38.432 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:38.432+0000 7ffb9e30c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-22T01:13:38.530 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:13:38.530+0000 7f9acd1de7c0 -1 Falling back to public interface 2024-09-22T01:13:38.733 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:13:38.734+0000 7f5ba98c37c0 -1 Falling back to public interface 2024-09-22T01:13:38.800 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:38.802+0000 7f6247d2b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T01:13:38.812 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:13:38.812+0000 7fc6161147c0 -1 Falling back to public interface 2024-09-22T01:13:38.820 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:13:38.820+0000 7f297859a7c0 -1 Falling back to public interface 2024-09-22T01:13:38.824 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:13:38.824+0000 7fe43c7677c0 -1 Falling back to public interface 2024-09-22T01:13:38.845 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:13:38.846+0000 7f6fafd0d7c0 -1 Falling back to public interface 2024-09-22T01:13:38.865 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:13:38.866+0000 7f42098d57c0 -1 Falling back to public interface 2024-09-22T01:13:38.881 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:38.882+0000 7f6247d2b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T01:13:39.034 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.032+0000 7ffb9e30c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-22T01:13:39.063 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:39.062+0000 7f6247d2b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T01:13:39.085 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:13:39.084+0000 7f1fc0b2f7c0 -1 Falling back to public interface 2024-09-22T01:13:39.110 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.108+0000 7ffb9e30c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-22T01:13:39.136 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:39.138+0000 7f6247d2b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T01:13:39.292 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.292+0000 7ffb9e30c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-22T01:13:39.365 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.364+0000 7ffb9e30c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-22T01:13:39.426 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:39.426+0000 7f6247d2b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T01:13:39.488 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:39.490+0000 7f6247d2b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T01:13:39.661 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.660+0000 7ffb9e30c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-22T01:13:39.723 INFO:tasks.ceph.mgr.y.smithi096.stderr:2024-09-22T01:13:39.720+0000 7ffb9e30c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-22T01:13:40.538 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:40.538+0000 7f61ee960640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-22T01:13:40.538 INFO:tasks.ceph.mgr.x.smithi150.stderr:2024-09-22T01:13:40.538+0000 7f61ee960640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-22T01:13:42.904 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:13:42.906+0000 7f5ba98c37c0 -1 osd.7 0 log_to_monitors true 2024-09-22T01:13:43.028 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:13:43.030+0000 7f9acd1de7c0 -1 osd.4 0 log_to_monitors true 2024-09-22T01:13:43.195 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:13:43.194+0000 7f42098d57c0 -1 osd.5 0 log_to_monitors true 2024-09-22T01:13:43.392 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:13:43.394+0000 7f6fafd0d7c0 -1 osd.6 0 log_to_monitors true 2024-09-22T01:13:43.399 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:13:43.400+0000 7fc6161147c0 -1 osd.0 0 log_to_monitors true 2024-09-22T01:13:43.611 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:13:43.608+0000 7f297859a7c0 -1 osd.3 0 log_to_monitors true 2024-09-22T01:13:43.717 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:13:43.716+0000 7fe43c7677c0 -1 osd.2 0 log_to_monitors true 2024-09-22T01:13:43.791 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:13:43.792+0000 7f1fc0b2f7c0 -1 osd.1 0 log_to_monitors true 2024-09-22T01:13:45.178 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:45.506 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-22T01:13:45.506 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":12,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:44.536768+0000","last_up_change":"0.000000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:45.524 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-22T01:13:45.678 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:13:45.676+0000 7fc61189a640 -1 osd.0 0 waiting for initial osdmap 2024-09-22T01:13:45.749 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:13:45.748+0000 7fc609ee1640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:46.899 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:13:46.898+0000 7f9ac8964640 -1 osd.4 0 waiting for initial osdmap 2024-09-22T01:13:46.913 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:13:46.914+0000 7f6fab493640 -1 osd.6 0 waiting for initial osdmap 2024-09-22T01:13:46.918 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:13:46.918+0000 7f5ba607e640 -1 osd.7 0 waiting for initial osdmap 2024-09-22T01:13:46.926 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:13:46.926+0000 7f420505b640 -1 osd.5 0 waiting for initial osdmap 2024-09-22T01:13:46.932 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:13:46.934+0000 7f9ac0fab640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:46.940 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:13:46.942+0000 7f5b9d690640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:46.952 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:13:46.954+0000 7f6fa3ada640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:46.952 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:13:46.954+0000 7f41fd6a2640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:47.505 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:13:47.504+0000 7f2973d20640 -1 osd.3 0 waiting for initial osdmap 2024-09-22T01:13:47.532 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:13:47.532+0000 7f296c367640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:47.555 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:13:47.556+0000 7fe437eed640 -1 osd.2 0 waiting for initial osdmap 2024-09-22T01:13:47.571 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:13:47.572+0000 7f1fbd2ea640 -1 osd.1 0 waiting for initial osdmap 2024-09-22T01:13:47.596 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:13:47.596+0000 7fe430534640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:47.612 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:13:47.612+0000 7f1fb48fc640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-22T01:13:52.331 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:52.713 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-22T01:13:52.713 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":19,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:51.970973+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:52.737 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-22T01:13:52.737 INFO:tasks.ceph:Creating RBD pool 2024-09-22T01:13:52.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-22T01:13:54.000 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'rbd' created 2024-09-22T01:13:54.025 DEBUG:teuthology.orchestra.run.smithi096:> rbd --cluster ceph pool init rbd 2024-09-22T01:13:57.066 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-22T01:13:57.066 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-22T01:13:57.067 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-22T01:13:57.067 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-22T01:13:57.067 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-22T01:13:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:13:57.450 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1691056029},{"type":"v1","addr":"172.21.15.150:6833","nonce":1691056029}]},"active_addr":"172.21.15.150:6833/1691056029","active_change":"2024-09-22T01:13:39.511444+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2396988860}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":1785509670}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2332359600}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.150:0","nonce":2523468670}]}]} 2024-09-22T01:13:57.455 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-22T01:13:57.455 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T01:13:57.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:57.757 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:13:57.757 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":23,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:57.022096+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:13:53.081727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:57.776 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T01:13:57.776 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:13:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:13:58.074 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":23,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:13:57.022096+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:13:37.127631+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:13:53.081727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:13:58.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T01:13:58.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T01:13:58.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T01:13:58.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T01:13:58.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-22T01:13:58.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-22T01:13:58.096 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-22T01:13:58.096 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-22T01:13:58.350 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:13:58.358 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:13:58.366 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.366 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:13:58.368 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:13:58.374 INFO:teuthology.orchestra.run.smithi096.stdout:64424509444 2024-09-22T01:13:58.374 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:13:58.376 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.377 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:13:58.379 INFO:teuthology.orchestra.run.smithi096.stdout:64424509446 2024-09-22T01:13:58.379 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:13:58.388 INFO:teuthology.orchestra.run.smithi096.stdout:60129542150 2024-09-22T01:13:58.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:13:58.847 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-09-22T01:13:58.874 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.885 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-22T01:13:58.904 INFO:teuthology.orchestra.run.smithi096.stdout:64424509443 2024-09-22T01:13:58.912 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.913 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-22T01:13:58.917 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:13:58.928 INFO:teuthology.orchestra.run.smithi096.stdout:60129542148 2024-09-22T01:13:58.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.5 2024-09-22T01:13:58.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-22T01:13:58.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-09-22T01:13:58.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-09-22T01:13:58.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.0 2024-09-22T01:13:59.870 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:13:59.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:13:59.914 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:13:59.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:13:59.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:13:59.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:13:59.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:13:59.950 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:14:00.383 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.1 2024-09-22T01:14:00.407 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.424 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.2 2024-09-22T01:14:00.441 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.451 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-09-22T01:14:00.469 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.482 INFO:teuthology.orchestra.run.smithi096.stdout:64424509445 2024-09-22T01:14:00.490 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.490 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.495 INFO:teuthology.orchestra.run.smithi096.stdout:60129542150 2024-09-22T01:14:00.499 INFO:teuthology.orchestra.run.smithi096.stdout:64424509447 2024-09-22T01:14:00.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.5 2024-09-22T01:14:00.505 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.6 2024-09-22T01:14:00.519 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.3 2024-09-22T01:14:00.522 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.7 2024-09-22T01:14:00.522 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-22T01:14:00.526 DEBUG:teuthology.parallel:result is None 2024-09-22T01:14:00.526 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:14:00.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:00.839 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:00.839 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:00.856 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-22T01:13:59.522675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223412,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751308,"statfs":{"total":773094113280,"available":772865339392,"internally_reserved":0,"allocated":3063808,"data_stored":1816174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.523958"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.048636+0000","last_change":"2024-09-22T01:13:57.048636+0000","last_active":"2024-09-22T01:13:57.048636+0000","last_peered":"2024-09-22T01:13:57.048636+0000","last_clean":"2024-09-22T01:13:57.048636+0000","last_became_active":"2024-09-22T01:13:55.011809+0000","last_became_peered":"2024-09-22T01:13:55.011809+0000","last_unstale":"2024-09-22T01:13:57.048636+0000","last_undegraded":"2024-09-22T01:13:57.048636+0000","last_fullsized":"2024-09-22T01:13:57.048636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:50.173115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025558270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.058638+0000","last_change":"2024-09-22T01:13:57.058638+0000","last_active":"2024-09-22T01:13:57.058638+0000","last_peered":"2024-09-22T01:13:57.058638+0000","last_clean":"2024-09-22T01:13:57.058638+0000","last_became_active":"2024-09-22T01:13:55.015176+0000","last_became_peered":"2024-09-22T01:13:55.015176+0000","last_unstale":"2024-09-22T01:13:57.058638+0000","last_undegraded":"2024-09-22T01:13:57.058638+0000","last_fullsized":"2024-09-22T01:13:57.058638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:12.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076979700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047628+0000","last_change":"2024-09-22T01:13:57.047628+0000","last_active":"2024-09-22T01:13:57.047628+0000","last_peered":"2024-09-22T01:13:57.047628+0000","last_clean":"2024-09-22T01:13:57.047628+0000","last_became_active":"2024-09-22T01:13:55.011956+0000","last_became_peered":"2024-09-22T01:13:55.011956+0000","last_unstale":"2024-09-22T01:13:57.047628+0000","last_undegraded":"2024-09-22T01:13:57.047628+0000","last_fullsized":"2024-09-22T01:13:57.047628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:23.965872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001589554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.040204+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:13:57.040204+0000","last_peered":"2024-09-22T01:13:57.040204+0000","last_clean":"2024-09-22T01:13:57.040204+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:13:57.040204+0000","last_undegraded":"2024-09-22T01:13:57.040204+0000","last_fullsized":"2024-09-22T01:13:57.040204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.049787+0000","last_change":"2024-09-22T01:13:57.048430+0000","last_active":"2024-09-22T01:13:57.049787+0000","last_peered":"2024-09-22T01:13:57.049787+0000","last_clean":"2024-09-22T01:13:57.049787+0000","last_became_active":"2024-09-22T01:13:55.341485+0000","last_became_peered":"2024-09-22T01:13:55.341485+0000","last_unstale":"2024-09-22T01:13:57.049787+0000","last_undegraded":"2024-09-22T01:13:57.049787+0000","last_fullsized":"2024-09-22T01:13:57.049787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:13.422509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023477490000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.404403+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:13:57.404403+0000","last_peered":"2024-09-22T01:13:57.404403+0000","last_clean":"2024-09-22T01:13:57.404403+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:13:57.404403+0000","last_undegraded":"2024-09-22T01:13:57.404403+0000","last_fullsized":"2024-09-22T01:13:57.404403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047098+0000","last_change":"2024-09-22T01:13:57.047098+0000","last_active":"2024-09-22T01:13:57.047098+0000","last_peered":"2024-09-22T01:13:57.047098+0000","last_clean":"2024-09-22T01:13:57.047098+0000","last_became_active":"2024-09-22T01:13:55.012782+0000","last_became_peered":"2024-09-22T01:13:55.012782+0000","last_unstale":"2024-09-22T01:13:57.047098+0000","last_undegraded":"2024-09-22T01:13:57.047098+0000","last_fullsized":"2024-09-22T01:13:57.047098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:36:23.140317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010592679999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047000+0000","last_change":"2024-09-22T01:13:57.047000+0000","last_active":"2024-09-22T01:13:57.047000+0000","last_peered":"2024-09-22T01:13:57.047000+0000","last_clean":"2024-09-22T01:13:57.047000+0000","last_became_active":"2024-09-22T01:13:55.339790+0000","last_became_peered":"2024-09-22T01:13:55.339790+0000","last_unstale":"2024-09-22T01:13:57.047000+0000","last_undegraded":"2024-09-22T01:13:57.047000+0000","last_fullsized":"2024-09-22T01:13:57.047000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:41.594287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094219900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.041062+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:13:57.041062+0000","last_peered":"2024-09-22T01:13:57.041062+0000","last_clean":"2024-09-22T01:13:57.041062+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:13:57.041062+0000","last_undegraded":"2024-09-22T01:13:57.041062+0000","last_fullsized":"2024-09-22T01:13:57.041062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:00.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:01.127 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:01.143 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-22T01:13:59.522675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223412,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751308,"statfs":{"total":773094113280,"available":772865339392,"internally_reserved":0,"allocated":3063808,"data_stored":1816174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.523958"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.048636+0000","last_change":"2024-09-22T01:13:57.048636+0000","last_active":"2024-09-22T01:13:57.048636+0000","last_peered":"2024-09-22T01:13:57.048636+0000","last_clean":"2024-09-22T01:13:57.048636+0000","last_became_active":"2024-09-22T01:13:55.011809+0000","last_became_peered":"2024-09-22T01:13:55.011809+0000","last_unstale":"2024-09-22T01:13:57.048636+0000","last_undegraded":"2024-09-22T01:13:57.048636+0000","last_fullsized":"2024-09-22T01:13:57.048636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:50.173115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025558270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.058638+0000","last_change":"2024-09-22T01:13:57.058638+0000","last_active":"2024-09-22T01:13:57.058638+0000","last_peered":"2024-09-22T01:13:57.058638+0000","last_clean":"2024-09-22T01:13:57.058638+0000","last_became_active":"2024-09-22T01:13:55.015176+0000","last_became_peered":"2024-09-22T01:13:55.015176+0000","last_unstale":"2024-09-22T01:13:57.058638+0000","last_undegraded":"2024-09-22T01:13:57.058638+0000","last_fullsized":"2024-09-22T01:13:57.058638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:12.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076979700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047628+0000","last_change":"2024-09-22T01:13:57.047628+0000","last_active":"2024-09-22T01:13:57.047628+0000","last_peered":"2024-09-22T01:13:57.047628+0000","last_clean":"2024-09-22T01:13:57.047628+0000","last_became_active":"2024-09-22T01:13:55.011956+0000","last_became_peered":"2024-09-22T01:13:55.011956+0000","last_unstale":"2024-09-22T01:13:57.047628+0000","last_undegraded":"2024-09-22T01:13:57.047628+0000","last_fullsized":"2024-09-22T01:13:57.047628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:23.965872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001589554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.040204+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:13:57.040204+0000","last_peered":"2024-09-22T01:13:57.040204+0000","last_clean":"2024-09-22T01:13:57.040204+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:13:57.040204+0000","last_undegraded":"2024-09-22T01:13:57.040204+0000","last_fullsized":"2024-09-22T01:13:57.040204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.049787+0000","last_change":"2024-09-22T01:13:57.048430+0000","last_active":"2024-09-22T01:13:57.049787+0000","last_peered":"2024-09-22T01:13:57.049787+0000","last_clean":"2024-09-22T01:13:57.049787+0000","last_became_active":"2024-09-22T01:13:55.341485+0000","last_became_peered":"2024-09-22T01:13:55.341485+0000","last_unstale":"2024-09-22T01:13:57.049787+0000","last_undegraded":"2024-09-22T01:13:57.049787+0000","last_fullsized":"2024-09-22T01:13:57.049787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:13.422509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023477490000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.404403+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:13:57.404403+0000","last_peered":"2024-09-22T01:13:57.404403+0000","last_clean":"2024-09-22T01:13:57.404403+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:13:57.404403+0000","last_undegraded":"2024-09-22T01:13:57.404403+0000","last_fullsized":"2024-09-22T01:13:57.404403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047098+0000","last_change":"2024-09-22T01:13:57.047098+0000","last_active":"2024-09-22T01:13:57.047098+0000","last_peered":"2024-09-22T01:13:57.047098+0000","last_clean":"2024-09-22T01:13:57.047098+0000","last_became_active":"2024-09-22T01:13:55.012782+0000","last_became_peered":"2024-09-22T01:13:55.012782+0000","last_unstale":"2024-09-22T01:13:57.047098+0000","last_undegraded":"2024-09-22T01:13:57.047098+0000","last_fullsized":"2024-09-22T01:13:57.047098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:36:23.140317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010592679999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047000+0000","last_change":"2024-09-22T01:13:57.047000+0000","last_active":"2024-09-22T01:13:57.047000+0000","last_peered":"2024-09-22T01:13:57.047000+0000","last_clean":"2024-09-22T01:13:57.047000+0000","last_became_active":"2024-09-22T01:13:55.339790+0000","last_became_peered":"2024-09-22T01:13:55.339790+0000","last_unstale":"2024-09-22T01:13:57.047000+0000","last_undegraded":"2024-09-22T01:13:57.047000+0000","last_fullsized":"2024-09-22T01:13:57.047000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:41.594287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094219900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.041062+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:13:57.041062+0000","last_peered":"2024-09-22T01:13:57.041062+0000","last_clean":"2024-09-22T01:13:57.041062+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:13:57.041062+0000","last_undegraded":"2024-09-22T01:13:57.041062+0000","last_fullsized":"2024-09-22T01:13:57.041062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:01.144 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:14:01.144 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-22T01:14:01.144 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-22T01:14:01.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-22T01:14:01.452 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:01.453 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-22T01:14:01.470 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-22T01:14:01.470 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-22T01:14:01.487 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-22T01:14:01.487 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'stupid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-22T01:14:01.487 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-22T01:14:01.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-22T01:14:01.785 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-22T01:14:01.785 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:14:01.785 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:02.063 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:02.081 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-22T01:14:01.523282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223412,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751308,"statfs":{"total":773094113280,"available":772865339392,"internally_reserved":0,"allocated":3063808,"data_stored":1816174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.522077"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.048636+0000","last_change":"2024-09-22T01:13:57.048636+0000","last_active":"2024-09-22T01:13:57.048636+0000","last_peered":"2024-09-22T01:13:57.048636+0000","last_clean":"2024-09-22T01:13:57.048636+0000","last_became_active":"2024-09-22T01:13:55.011809+0000","last_became_peered":"2024-09-22T01:13:55.011809+0000","last_unstale":"2024-09-22T01:13:57.048636+0000","last_undegraded":"2024-09-22T01:13:57.048636+0000","last_fullsized":"2024-09-22T01:13:57.048636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:50.173115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025558270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.058638+0000","last_change":"2024-09-22T01:13:57.058638+0000","last_active":"2024-09-22T01:13:57.058638+0000","last_peered":"2024-09-22T01:13:57.058638+0000","last_clean":"2024-09-22T01:13:57.058638+0000","last_became_active":"2024-09-22T01:13:55.015176+0000","last_became_peered":"2024-09-22T01:13:55.015176+0000","last_unstale":"2024-09-22T01:13:57.058638+0000","last_undegraded":"2024-09-22T01:13:57.058638+0000","last_fullsized":"2024-09-22T01:13:57.058638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:12.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076979700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047628+0000","last_change":"2024-09-22T01:13:57.047628+0000","last_active":"2024-09-22T01:13:57.047628+0000","last_peered":"2024-09-22T01:13:57.047628+0000","last_clean":"2024-09-22T01:13:57.047628+0000","last_became_active":"2024-09-22T01:13:55.011956+0000","last_became_peered":"2024-09-22T01:13:55.011956+0000","last_unstale":"2024-09-22T01:13:57.047628+0000","last_undegraded":"2024-09-22T01:13:57.047628+0000","last_fullsized":"2024-09-22T01:13:57.047628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:23.965872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001589554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.040204+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:13:57.040204+0000","last_peered":"2024-09-22T01:13:57.040204+0000","last_clean":"2024-09-22T01:13:57.040204+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:13:57.040204+0000","last_undegraded":"2024-09-22T01:13:57.040204+0000","last_fullsized":"2024-09-22T01:13:57.040204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.049787+0000","last_change":"2024-09-22T01:13:57.048430+0000","last_active":"2024-09-22T01:13:57.049787+0000","last_peered":"2024-09-22T01:13:57.049787+0000","last_clean":"2024-09-22T01:13:57.049787+0000","last_became_active":"2024-09-22T01:13:55.341485+0000","last_became_peered":"2024-09-22T01:13:55.341485+0000","last_unstale":"2024-09-22T01:13:57.049787+0000","last_undegraded":"2024-09-22T01:13:57.049787+0000","last_fullsized":"2024-09-22T01:13:57.049787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:13.422509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023477490000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.404403+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:13:57.404403+0000","last_peered":"2024-09-22T01:13:57.404403+0000","last_clean":"2024-09-22T01:13:57.404403+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:13:57.404403+0000","last_undegraded":"2024-09-22T01:13:57.404403+0000","last_fullsized":"2024-09-22T01:13:57.404403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047098+0000","last_change":"2024-09-22T01:13:57.047098+0000","last_active":"2024-09-22T01:13:57.047098+0000","last_peered":"2024-09-22T01:13:57.047098+0000","last_clean":"2024-09-22T01:13:57.047098+0000","last_became_active":"2024-09-22T01:13:55.012782+0000","last_became_peered":"2024-09-22T01:13:55.012782+0000","last_unstale":"2024-09-22T01:13:57.047098+0000","last_undegraded":"2024-09-22T01:13:57.047098+0000","last_fullsized":"2024-09-22T01:13:57.047098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:36:23.140317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010592679999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047000+0000","last_change":"2024-09-22T01:13:57.047000+0000","last_active":"2024-09-22T01:13:57.047000+0000","last_peered":"2024-09-22T01:13:57.047000+0000","last_clean":"2024-09-22T01:13:57.047000+0000","last_became_active":"2024-09-22T01:13:55.339790+0000","last_became_peered":"2024-09-22T01:13:55.339790+0000","last_unstale":"2024-09-22T01:13:57.047000+0000","last_undegraded":"2024-09-22T01:13:57.047000+0000","last_fullsized":"2024-09-22T01:13:57.047000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:41.594287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094219900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.041062+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:13:57.041062+0000","last_peered":"2024-09-22T01:13:57.041062+0000","last_clean":"2024-09-22T01:13:57.041062+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:13:57.041062+0000","last_undegraded":"2024-09-22T01:13:57.041062+0000","last_fullsized":"2024-09-22T01:13:57.041062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:02.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:02.360 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:02.360 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:02.376 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-22T01:14:01.523282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223412,"kb_used_data":2992,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751308,"statfs":{"total":773094113280,"available":772865339392,"internally_reserved":0,"allocated":3063808,"data_stored":1816174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.522077"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.048636+0000","last_change":"2024-09-22T01:13:57.048636+0000","last_active":"2024-09-22T01:13:57.048636+0000","last_peered":"2024-09-22T01:13:57.048636+0000","last_clean":"2024-09-22T01:13:57.048636+0000","last_became_active":"2024-09-22T01:13:55.011809+0000","last_became_peered":"2024-09-22T01:13:55.011809+0000","last_unstale":"2024-09-22T01:13:57.048636+0000","last_undegraded":"2024-09-22T01:13:57.048636+0000","last_fullsized":"2024-09-22T01:13:57.048636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:47:50.173115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025558270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.058638+0000","last_change":"2024-09-22T01:13:57.058638+0000","last_active":"2024-09-22T01:13:57.058638+0000","last_peered":"2024-09-22T01:13:57.058638+0000","last_clean":"2024-09-22T01:13:57.058638+0000","last_became_active":"2024-09-22T01:13:55.015176+0000","last_became_peered":"2024-09-22T01:13:55.015176+0000","last_unstale":"2024-09-22T01:13:57.058638+0000","last_undegraded":"2024-09-22T01:13:57.058638+0000","last_fullsized":"2024-09-22T01:13:57.058638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:12.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076979700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047628+0000","last_change":"2024-09-22T01:13:57.047628+0000","last_active":"2024-09-22T01:13:57.047628+0000","last_peered":"2024-09-22T01:13:57.047628+0000","last_clean":"2024-09-22T01:13:57.047628+0000","last_became_active":"2024-09-22T01:13:55.011956+0000","last_became_peered":"2024-09-22T01:13:55.011956+0000","last_unstale":"2024-09-22T01:13:57.047628+0000","last_undegraded":"2024-09-22T01:13:57.047628+0000","last_fullsized":"2024-09-22T01:13:57.047628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:23.965872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001589554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.040204+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:13:57.040204+0000","last_peered":"2024-09-22T01:13:57.040204+0000","last_clean":"2024-09-22T01:13:57.040204+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:13:57.040204+0000","last_undegraded":"2024-09-22T01:13:57.040204+0000","last_fullsized":"2024-09-22T01:13:57.040204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.049787+0000","last_change":"2024-09-22T01:13:57.048430+0000","last_active":"2024-09-22T01:13:57.049787+0000","last_peered":"2024-09-22T01:13:57.049787+0000","last_clean":"2024-09-22T01:13:57.049787+0000","last_became_active":"2024-09-22T01:13:55.341485+0000","last_became_peered":"2024-09-22T01:13:55.341485+0000","last_unstale":"2024-09-22T01:13:57.049787+0000","last_undegraded":"2024-09-22T01:13:57.049787+0000","last_fullsized":"2024-09-22T01:13:57.049787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:38:13.422509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023477490000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.404403+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:13:57.404403+0000","last_peered":"2024-09-22T01:13:57.404403+0000","last_clean":"2024-09-22T01:13:57.404403+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:13:57.404403+0000","last_undegraded":"2024-09-22T01:13:57.404403+0000","last_fullsized":"2024-09-22T01:13:57.404403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047098+0000","last_change":"2024-09-22T01:13:57.047098+0000","last_active":"2024-09-22T01:13:57.047098+0000","last_peered":"2024-09-22T01:13:57.047098+0000","last_clean":"2024-09-22T01:13:57.047098+0000","last_became_active":"2024-09-22T01:13:55.012782+0000","last_became_peered":"2024-09-22T01:13:55.012782+0000","last_unstale":"2024-09-22T01:13:57.047098+0000","last_undegraded":"2024-09-22T01:13:57.047098+0000","last_fullsized":"2024-09-22T01:13:57.047098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:36:23.140317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010592679999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.047000+0000","last_change":"2024-09-22T01:13:57.047000+0000","last_active":"2024-09-22T01:13:57.047000+0000","last_peered":"2024-09-22T01:13:57.047000+0000","last_clean":"2024-09-22T01:13:57.047000+0000","last_became_active":"2024-09-22T01:13:55.339790+0000","last_became_peered":"2024-09-22T01:13:55.339790+0000","last_unstale":"2024-09-22T01:13:57.047000+0000","last_undegraded":"2024-09-22T01:13:57.047000+0000","last_fullsized":"2024-09-22T01:13:57.047000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:41.594287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094219900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-22T01:13:57.041062+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:13:57.041062+0000","last_peered":"2024-09-22T01:13:57.041062+0000","last_clean":"2024-09-22T01:13:57.041062+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:13:57.041062+0000","last_undegraded":"2024-09-22T01:13:57.041062+0000","last_fullsized":"2024-09-22T01:13:57.041062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":669793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":79425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:02.377 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:14:02.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-22T01:14:02.670 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 23 2024-09-22T01:14:02.670 INFO:teuthology.orchestra.run.smithi096.stdout:fsid e212681a-0600-4f32-ab96-8ca46d37e9a4 2024-09-22T01:14:02.670 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-22T01:13:34.805751+0000 2024-09-22T01:14:02.670 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-09-22T01:13:57.022096+0000 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 5 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client jewel 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-22T01:14:02.671 INFO:teuthology.orchestra.run.smithi096.stdout: removed_snaps_queue [2~1] 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 8 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/3838492416,v1:172.21.15.96:6801/3838492416] [v2:172.21.15.96:6802/3838492416,v1:172.21.15.96:6803/3838492416] exists,up a0e12724-c6a8-43ce-aad2-d3043c2f23d9 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/1053715553,v1:172.21.15.96:6825/1053715553] [v2:172.21.15.96:6826/1053715553,v1:172.21.15.96:6827/1053715553] exists,up bd73805d-596e-40f0-ba54-69ca84175c51 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/860290285,v1:172.21.15.96:6817/860290285] [v2:172.21.15.96:6818/860290285,v1:172.21.15.96:6819/860290285] exists,up 90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/740986778,v1:172.21.15.96:6809/740986778] [v2:172.21.15.96:6810/740986778,v1:172.21.15.96:6811/740986778] exists,up eed7a22f-aa64-4dad-9151-8fc05271275e 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/787691336,v1:172.21.15.150:6801/787691336] [v2:172.21.15.150:6802/787691336,v1:172.21.15.150:6803/787691336] exists,up 14ed2997-6237-4ca6-a17b-0c0ae87f0645 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/721780330,v1:172.21.15.150:6825/721780330] [v2:172.21.15.150:6826/721780330,v1:172.21.15.150:6827/721780330] exists,up 6ecd111f-9cc2-45a6-92f3-67f73bacfdad 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/2103163729,v1:172.21.15.150:6817/2103163729] [v2:172.21.15.150:6818/2103163729,v1:172.21.15.150:6819/2103163729] exists,up 5032e618-100c-4318-9825-3e90a3c4b25e 2024-09-22T01:14:02.672 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/1517191652,v1:172.21.15.150:6809/1517191652] [v2:172.21.15.150:6810/1517191652,v1:172.21.15.150:6811/1517191652] exists,up 020cdc04-e2da-4af5-b6d9-674e88425186 2024-09-22T01:14:02.688 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/3838492416,v1:172.21.15.96:6801/3838492416] [v2:172.21.15.96:6802/3838492416,v1:172.21.15.96:6803/3838492416] exists,up a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'osd.1 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/1053715553,v1:172.21.15.96:6825/1053715553] [v2:172.21.15.96:6826/1053715553,v1:172.21.15.96:6827/1053715553] exists,up bd73805d-596e-40f0-ba54-69ca84175c51', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/860290285,v1:172.21.15.96:6817/860290285] [v2:172.21.15.96:6818/860290285,v1:172.21.15.96:6819/860290285] exists,up 90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/740986778,v1:172.21.15.96:6809/740986778] [v2:172.21.15.96:6810/740986778,v1:172.21.15.96:6811/740986778] exists,up eed7a22f-aa64-4dad-9151-8fc05271275e', 'osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/787691336,v1:172.21.15.150:6801/787691336] [v2:172.21.15.150:6802/787691336,v1:172.21.15.150:6803/787691336] exists,up 14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/721780330,v1:172.21.15.150:6825/721780330] [v2:172.21.15.150:6826/721780330,v1:172.21.15.150:6827/721780330] exists,up 6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/2103163729,v1:172.21.15.150:6817/2103163729] [v2:172.21.15.150:6818/2103163729,v1:172.21.15.150:6819/2103163729] exists,up 5032e618-100c-4318-9825-3e90a3c4b25e', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/1517191652,v1:172.21.15.150:6809/1517191652] [v2:172.21.15.150:6810/1517191652,v1:172.21.15.150:6811/1517191652] exists,up 020cdc04-e2da-4af5-b6d9-674e88425186'] 2024-09-22T01:14:02.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-22T01:14:02.800 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mon.a", 2024-09-22T01:14:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster": "ceph", 2024-09-22T01:14:02.800 INFO:teuthology.orchestra.run.smithi096.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "admin_socket_mode": "", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_client_required": "cephx, none", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_cluster_required": "cephx", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_debug": "true", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_service_required": "cephx", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_supported": "cephx", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio": "true", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_poll_ms": "250", 2024-09-22T01:14:02.801 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_aio_reap_max": "16", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_async_discard": "false", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_block_size": "4096", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio": "false", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_debug_inflight_ios": "false", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_enable_discard": "false", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_flock_retry": "3", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_inject_crash": "0", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-22T01:14:02.802 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring_hipri": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bdev_type": "", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_alloc_size": "1048576", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_allocator": "hybrid", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_buffered_io": "true", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_for_zeros": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-22T01:14:02.803 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_compact_log_sync": "false", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_max_log_runway": "4194304", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_max_prefetch": "1048576", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_min_flush_size": "524288", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_min_log_runway": "1048576", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_replay_recovery": "false", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_sync_write": "false", 2024-09-22T01:14:02.804 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_allocation_from_file": "true", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_allocator": "hybrid", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bdev_label_multi": "true", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-22T01:14:02.805 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_create": "true", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_create": "false", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_path": "", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_db_size": "0", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_path": "", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_preallocate_file": "false", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_size": "107374182400", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_create": "false", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_path": "", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_block_wal_size": "100663296", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs": "true", 2024-09-22T01:14:02.806 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_autotune": "true", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-22T01:14:02.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size": "0", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_type": "2q", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_clone_cow": "true", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-22T01:14:02.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_mode": "none", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_csum_type": "crc32c", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_freelist": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-22T01:14:02.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_small_allocations": "0", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_default_buffered_read": "true", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_default_buffered_write": "false", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-22T01:14:02.810 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fail_eio": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mount": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_umount": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-22T01:14:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_ignore_data_csum": "false", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_log_op_age": "5.000000", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_alloc_size": "0", 2024-09-22T01:14:02.812 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size": "0", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_max_deferred_txc": "32", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size": "0", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_nid_prealloc": "1024", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_retry_disk_reads": "3", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_cf": "true", 2024-09-22T01:14:02.813 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_spdk_mem": "512", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-22T01:14:02.814 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_tracing": "false", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_zero_block_detection": "false", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-22T01:14:02.815 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_cluster_require_signatures": "false", 2024-09-22T01:14:02.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_cluster_require_version": "2", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_require_signatures": "false", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_require_version": "2", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_service_require_signatures": "false", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_service_require_version": "2", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "cephx_sign_messages": "true", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "chdir": "", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_acl_type": "", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_asio_thread_count": "2", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_cache_mid": "0.750000", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_cache_size": "16384", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_caps_release_delay": "5", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_check_pool_perm": "true", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-22T01:14:02.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_force_sync_read": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_getattr_caps": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_inject_features": "", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_debug_inject_tick_delay": "0", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_die_on_failed_remount": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_dirsize_rbytes": "true", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_force_lazyio": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_fs": "", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_inject_release_failure": "false", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_max_inline_size": "4096", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mds_namespace": "", 2024-09-22T01:14:02.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_metadata": "", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_gid": "-1", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_timeout": "300", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mount_uid": "-1", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_mountpoint": "/", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_notify_timeout": "10", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc": "true", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_dirty": "104857600", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_max_objects": "1000", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_size": "209715200", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_oc_target_dirty": "8388608", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_permissions": "true", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_quota": "true", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_quota_df": "true", 2024-09-22T01:14:02.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_max_bytes": "0", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_max_periods": "4", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_readahead_min": "131072", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_reconnect_stale": "false", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_shutdown_timeout": "30", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_snapdir": ".snap", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_tick_interval": "1", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_trace": "", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_try_dentry_invalidate": "false", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_use_faked_inos": "false", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_use_random_mds": "false", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog": "false", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_graylog_port": "12201", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_monitors": "default=true", 2024-09-22T01:14:02.820 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog": "false", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "clog_to_syslog_level": "info", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "-", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_network": "", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_network_interface": "", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_isal": "false", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_level": "5", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zlib_winsize": "-15", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "compressor_zstd_level": "1", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_alien_op_num_threads": "6", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-22T01:14:02.821 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_osd_stat_interval": "0", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_seastar_cpu_cores": "", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crimson_seastar_num_threads": "0", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location": "", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location_hook": "", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_location_hook_timeout": "10", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "daemonize": "false", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "daos_pool": "tank", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_alienstore": "0/5", 2024-09-22T01:14:02.822 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_allow_any_pool_priority": "false", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asok": "1/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asok_assert_abort": "false", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_asserts_on_shutdown": "true", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_auth": "20/20", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bdev": "1/3", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bluefs": "1/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_bluestore": "1/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_buffer": "0/1", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_ceph_exporter": "1/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_cephfs_mirror": "0/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_cephsqlite": "0/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_client": "0/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_compressor": "1/5", 2024-09-22T01:14:02.823 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_context": "0/1", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_crimson_interrupt": "0/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_crush": "1/1", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_crypto": "1/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_cyanstore": "0/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_deliberately_leak_memory": "false", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_disable_randomized_ping": "false", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_dpdk": "1/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_eventtrace": "1/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_filer": "0/1", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_filestore": "1/3", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_finisher": "1/1", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_fuse": "1/5", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_heartbeat_testing_span": "0", 2024-09-22T01:14:02.824 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_heartbeatmap": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_javaclient": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_journal": "1/3", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_journaler": "0/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_kstore": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_lockdep": "0/1", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mclock": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_balancer": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_locker": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_log": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_log_expire": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_migrator": "1/5", 2024-09-22T01:14:02.825 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mds_quiesce": "3/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_memstore": "1/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mgr": "2/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mgrc": "1/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_mon": "20/20", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_monc": "0/10", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_ms": "1/1", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_none": "0/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objclass": "0/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objectcacher": "0/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_objecter": "0/1", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_optracker": "0/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_osd": "1/5", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_paxos": "20/20", 2024-09-22T01:14:02.826 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_perfcounter": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_prioritycache": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rados": "0/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd": "0/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_mirror": "0/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_pwl": "0/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rbd_replay": "0/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_refs": "0/0", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_reserver": "1/1", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_access": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_datacache": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_dbstore": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_flight": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-22T01:14:02.827 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rgw_sync": "1/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_rocksdb": "4/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_backref": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_cache": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_cleaner": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_device": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_epm": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_journal": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_lba": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_odata": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_omap": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_onode": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_t": "0/5", 2024-09-22T01:14:02.828 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_seastore_tm": "0/5", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_striper": "0/1", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_test": "0/5", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_throttle": "1/1", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_timer": "0/1", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_tp": "0/5", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "debug_trace": "1/5", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "device_failure_prediction_mode": "none", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_graylog": "false", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_journald": "false", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_stderr": "true", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "err_to_syslog": "false", 2024-09-22T01:14:02.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event_tracing": "false", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_addr": "0.0.0.0", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_http_port": "9926", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_prio_limit": "5", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_sort_metrics": "true", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "exporter_stats_period": "5", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "fake_statfs_for_testing": "0", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "fatal_signal_handlers": "true", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "filer_max_purge_ops": "10", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "filer_max_truncate_ops": "128", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_apply_finisher_threads": "1", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_blackhole": "false", 2024-09-22T01:14:02.830 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_btrfs_clone_range": "true", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_btrfs_snap": "true", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_caller_concurrency": "10", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_collect_device_partition_information": "true", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_commit_timeout": "600.000000", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_inject_read_err": "false", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_omap_check": "false", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_debug_verify_split": "false", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_dump_file": "", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fadvise": "true", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fail_eio": "true", 2024-09-22T01:14:02.831 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fd_cache_shards": "16", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fd_cache_size": "128", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fiemap": "false", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fiemap_threshold": "4096", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_inject_stall": "0", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_parallel": "false", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_trailing": "false", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_journal_writeahead": "false", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_kill_at": "0", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-22T01:14:02.832 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs": "0", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size": "0", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_merge_threshold": "-10", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_odsync_write": "false", 2024-09-22T01:14:02.833 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_backend": "rocksdb", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_backend_path": "", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_thread_timeout": "60", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_op_threads": "2", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_punch_hole": "false", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-22T01:14:02.834 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_queue_max_ops": "50", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_seek_data_hole": "false", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_sloppy_crc": "false", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_splice": "false", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_split_multiple": "2", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_split_rand_factor": "20", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_update_to": "1000", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-22T01:14:02.835 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_enable": "true", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_xfs_extsize": "false", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "filestore_zfs_snap": "false", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "fio_dir": "/tmp/fio", 2024-09-22T01:14:02.836 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_allow_other": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_atomic_o_trunc": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_big_writes": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_debug": "false", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_default_permissions": "false", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_disable_pagecache": "false", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_max_write": "0", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_multithreaded": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_require_active_mds": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_set_user_groups": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_move": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_read": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_splice_write": "true", 2024-09-22T01:14:02.837 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "fuse_use_invalidate_cb": "true", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "gss_target_name": "ceph", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_file": "", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_inject_failure": "0", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_interval": "5", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "host": "smithi096", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-22T01:14:02.838 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "inject_early_sigterm": "false", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "jaeger_agent_port": "6799", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "jaeger_tracing_enable": "false", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_aio": "true", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_align_min_size": "65536", 2024-09-22T01:14:02.839 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_block_align": "true", 2024-09-22T01:14:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_block_size": "4096", 2024-09-22T01:14:02.840 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_dio": "true", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_discard": "false", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_force_aio": "false", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_ignore_corruption": "false", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_max_write_bytes": "10485760", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_max_write_entries": "100", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_replay_from": "0", 2024-09-22T01:14:02.842 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_write_header_frequency": "0", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journal_zero_on_create": "false", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_prefetch_periods": "10", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_prezero_periods": "5", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "journaler_write_head_interval": "15", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "keyfile": "", 2024-09-22T01:14:02.843 INFO:teuthology.orchestra.run.smithi096.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_backend": "rocksdb", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_default_stripe_size": "65536", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_fsck_on_mount": "false", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_max_bytes": "67108864", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_max_ops": "512", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_nid_prealloc": "1024", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_onode_map_size": "1024", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_sync_submit_transaction": "false", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "kstore_sync_transaction": "false", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "librados_thread_count": "2", 2024-09-22T01:14:02.844 INFO:teuthology.orchestra.run.smithi096.stdout: "lockdep": "false", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "lockdep_force_backtrace": "false", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_coarse_timestamps": "true", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_flush_on_exit": "false", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_graylog_host": "127.0.0.1", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_graylog_port": "12201", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_max_new": "1000", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_max_recent": "10000", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_stderr_prefix": "", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_stop_at_utilization": "0.970000", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": "true", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_graylog": "false", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_journald": "false", 2024-09-22T01:14:02.845 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_stderr": "false", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_syslog": "false", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max_rotating_auth_attempts": "10", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_action_on_write_error": "1", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_alternate_name_max": "8192", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_asio_thread_count": "2", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_export_pin": "true", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_dirs": "true", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_interval": "5", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_interval": "10", 2024-09-22T01:14:02.846 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_max": "-1", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_max_until": "-1", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_merge_size": "50", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_midchunk": "0.300000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_min_start": "0.200000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_minchunk": "0.001000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_mode": "0", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_need_max": "1.200000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_need_min": "0.800000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_overload_epochs": "2", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_bits": "3", 2024-09-22T01:14:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_size": "10000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_target_decay": "10.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_grace": "15.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_interval": "4.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_mid": "0.700000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_quiesce_delay": "0", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-22T01:14:02.848 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_release_free_interval": "10", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_reservation": "0.050000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_interval": "1", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache_trim_threshold": "262144", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_prealloc_inos": "1000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_connect_bootstrapping": "false", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_damage_table_max_entries": "10000", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_auth_pins": "false", 2024-09-22T01:14:02.849 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_frag": "false", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_scatterstat": "false", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_debug_subtrees": "false", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_decay_halflife": "5.000000", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_default_dir_hash": "2", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_defer_session_stale": "true", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_deny_all_reconnect": "false", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_keys_per_op": "16384", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_max_commit_size": "10", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_max_entries": "0", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dir_prefetch": "true", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-22T01:14:02.850 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_on_map": "false", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_early_reply": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_enable_op_tracker": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_enforce_unique_name": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_random": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-22T01:14:02.851 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_health_summarize_threshold": "10", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_health_dummy": "false", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_migrator_session_race": "false", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_join_fs": "", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_journal_format": "1", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-22T01:14:02.852 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_create_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_export_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_import_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_expire_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_journal_replay_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_link_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_mdstable_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_openc_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_rename_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_kill_shutdown_at": "0", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_event_large_threshold": "524288", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_events_per_segment": "1024", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-22T01:14:02.853 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_max_events": "-1", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_max_segments": "128", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_pause": "false", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_segment_size": "0", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_trim_threshold": "128", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log_warn_factor": "2.000000", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_caps_per_client": "1048576", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_completed_flushes": "100000", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_completed_requests": "100000", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_export_size": "20971520", 2024-09-22T01:14:02.854 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_file_recover": "32", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_files": "64", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_ops": "8192", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_max_snaps_per_dir": "100", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_metrics_update_interval": "2", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_min_caps_per_client": "100", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_min_caps_working_set": "10000", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_numa_node": "-1", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_complaint_time": "30.000000", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_duration": "600", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_size": "20", 2024-09-22T01:14:02.855 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_slow_op_size": "20", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_op_log_threshold": "5", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_ping_grace": "15", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_ping_interval": "5", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_caps": "30000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_recall_warning_threshold": "262144", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_replay_interval": "1.000000", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-22T01:14:02.856 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_root_ino_gid": "0", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_root_ino_uid": "0", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_shutdown_check": "0", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_skip_ino": "0", 2024-09-22T01:14:02.857 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_max_uid": "4294967294", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_min_uid": "0", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_snap_rstat": "false", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_standby_replay_damaged": "false", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_symlink_recovery": "true", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_thrash_exports": "0", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_thrash_fragments": "0", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_tick_interval": "5.000000", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_valgrind_exit": "false", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_verify_backtrace": "1", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_verify_scatter": "false", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-22T01:14:02.858 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_wipe_sessions": "false", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool_debug": "false", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_device_bytes": "1073741824", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_page_set": "false", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "memstore_page_size": "65536", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_bytes": "134217728", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_messages": "512", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_disabled_modules": "", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_enable_op_tracker": "true", 2024-09-22T01:14:02.859 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_max_pg_num_change": "128", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mds_bytes": "134217728", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mds_messages": "128", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mon_bytes": "134217728", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_mon_messages": "128", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_history_duration": "600", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_history_size": "20", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_op_log_threshold": "5", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_osd_bytes": "536870912", 2024-09-22T01:14:02.860 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_osd_messages": "8192", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_pool": "true", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_standby_modules": "true", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_stats_period": "5", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_stats_threshold": "5", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_tick_period": "2", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_allow_pool_delete": "true", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_allow_pool_size_one": "true", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_auth_validate_all_caps": "true", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_bytes": "104857600", 2024-09-22T01:14:02.861 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_directed_command_retry": "5", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_hunt_parallel": "3", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_log_interval": "1.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_ping_interval": "10.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_client_target_rank": "-1", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-22T01:14:02.862 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_file_level": "debug", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_file": "true", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_journald": "false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_bootstrap": "false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_start": "false", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_compact_on_trim": "true", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-22T01:14:02.863 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_cpu_threads": "4", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_crush_min_required_version": "hammer", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_daemon_bytes": "419430400", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_avail_crit": "5", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_avail_warn": "5", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_data_size_warn": "16106127360", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_json": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_dump_transactions": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_extra_checks": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-22T01:14:02.864 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_reef": "false", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_no_require_squid": "false", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_down_mkfs_grace": "120", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_down_uptime_grace": "60", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_election_default_strategy": "1", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_election_timeout": "5.000000", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ping_divisor": "2", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_enable_op_tracker": "true", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_fake_pool_delete": "false", 2024-09-22T01:14:02.865 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_force_quorum_join": "false", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_globalid_prealloc": "10000", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_detail_to_clog": "false", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_log_update_period": "5", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_max_detail": "50", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog": "true", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog_interval": "600", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_host": "172.21.15.96,[v2:172.21.15.96:3301,v1:172.21.15.96:6790],172.21.15.150", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_host_override": "", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_initial_members": "", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-22T01:14:02.866 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease": "5.000000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_full_interval": "50", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_max": "10000", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_log_max_summary": "50", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_log_epochs": "500", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-22T01:14:02.867 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_osd": "10000", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_pg_per_osd": "10000", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_pool_pg_num": "65536", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_force_trim_to": "0", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mds_skip_sanity": "false", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_memory_autotune": "true", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_memory_target": "2147483648", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_beacon_grace": "30", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_digest_period": "5", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_inactive_grace": "60", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-22T01:14:02.868 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_min_osdmap_epochs": "500", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_complaint_time": "30", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_duration": "600", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_size": "20", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_slow_op_size": "20", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_op_log_threshold": "5", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_in": "false", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_cache_size": "500", 2024-09-22T01:14:02.869 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_down_out_interval": "600", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_force_trim_to": "0", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-22T01:14:02.870 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_down_reporters": "2", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_report_timeout": "900", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-22T01:14:02.871 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_stuck_threshold": "60", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_probe_timeout": "2.000000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_max_change": "0.050000", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_max_osds": "4", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-22T01:14:02.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_interval": "86400", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_max_keys": "100", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_scrub_timeout": "300", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_session_timeout": "300", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_smart_report_timeout": "5", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stat_smooth_intervals": "6", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_pool_min_size": "2", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_pool_size": "4", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-22T01:14:02.873 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_debug": "false", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_provider_kill_at": "0", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_requester_kill_at": "0", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_sync_timeout": "60.000000", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_target_pg_per_osd": "100", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_tick_interval": "5", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_timecheck_interval": "300.000000", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_use_min_delay_socket": "false", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_older_version_delay": "604800", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-22T01:14:02.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_misplaced": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_older_version": "true", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-22T01:14:02.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "monmap": "", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "motr_tracing_enabled": "false", 2024-09-22T01:14:02.876 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_op_threads": "3", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_cm": "false", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_device_name": "", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_dscp": "96", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_local_gid": "", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_port_num": "1", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-22T01:14:02.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_sl": "3", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_support_srq": "true", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_rdma_type": "ib", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_async_reap_threshold": "5", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_before_connect": "false", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_ipv4": "true", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_ipv6": "false", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_msgr1": "true", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_msgr2": "true", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_port_max": "7568", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_port_min": "6800", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_retry_count": "3", 2024-09-22T01:14:02.878 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_bind_retry_delay": "5", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_client": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mds": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mgr": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_mon": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_blackhole_osd": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_client_mode": "crc secure", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_cluster_mode": "crc secure", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_cluster_type": "", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_compress_secure": "false", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_connection_idle_timeout": "900", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_connection_ready_timeout": "10", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_crc_data": "true", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_crc_header": "true", 2024-09-22T01:14:02.879 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_bad_msg": "false", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_bug": "true", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_old_message": "true", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_skipped_message": "false", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_coremask": "0xF", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_enable_tso": "true", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hugepages": "", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-22T01:14:02.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_lro": "true", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_memory_channel": "4", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_pmd": "", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_port_id": "0", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_dump_on_send": "false", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_initial_backoff": "0.200000", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_max": "1.000000", 2024-09-22T01:14:02.881 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_delay_type": "", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_network_congestion": "0", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_inject_socket_failures": "5000", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_learn_addr_from_peer": "true", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_max_accept_failures": "4", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_max_backoff": "15.000000", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_client_mode": "secure crc", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_mon_service_mode": "secure crc", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compress_min_size": "1024", 2024-09-22T01:14:02.882 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compress_mode": "none", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_pq_min_cost": "65536", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_public_type": "", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_service_mode": "crc secure", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_listen_backlog": "512", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_nodelay": "true", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_tcp_rcvbuf": "0", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "ms_type": "async+posix", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "no_config_file": "false", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_completion_locks_per_session": "32", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-22T01:14:02.883 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inflight_ops": "1024", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_tick_interval": "5.000000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter_timeout": "10.000000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objectstore_blackhole": "false", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "openssl_engine_opts": "", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_delay_time": "5.000000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_hist_halflife": "1000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_max_low_ops": "2", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_max_ops": "4", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-22T01:14:02.884 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_agent_slop": "0.020000", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_asio_thread_count": "2", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_async_recovery_min_cost": "100", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_scan_max": "512", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backfill_scan_min": "64", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_degraded": "false", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_peering": "false", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_backoff_on_unfound": "true", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_beacon_report_interval": "300", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_duration": "30", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-22T01:14:02.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_max_block_size": "67108864", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_blkin_trace_all": "false", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_check_for_log_corruption": "false", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_class_update_on_start": "true", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_message_cap": "256", 2024-09-22T01:14:02.886 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_message_size_cap": "524288000", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_op_priority": "63", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_client_watch_timeout": "30", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_command_max_records": "256", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_compact_on_start": "false", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_update_on_start": "true", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_crush_update_weight_set": "true", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-22T01:14:02.887 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_feed_pullee": "-1", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_misdirected_ops": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_no_acting_change": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_no_purge_strays": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_op_order": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_shutdown": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-22T01:14:02.888 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_trim_objects": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_snaps": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_keys": "1024", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-22T01:14:02.889 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_stride": "524288", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_default_notify_timeout": "30", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep": "0.000000", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_discard_disconnected_ops": "true", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_enable_op_tracker": "true", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-22T01:14:02.890 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_info": "true", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown": "true", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_function_tracing": "false", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_grace": "20", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_interval": "6", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_peers": "10", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_min_size": "2000", 2024-09-22T01:14:02.891 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_stale": "600", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_max_size": "100000", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_min_size": "1000", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_journal_size": "5120", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_kill_backfill_at": "0", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_cache_size": "50", 2024-09-22T01:14:02.892 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_dedup": "true", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_message_max": "40", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_map_share_max_epochs": "40", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_attr_name_len": "100", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_attr_size": "0", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_backfills": "1", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_markdown_count": "5", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_markdown_period": "600", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_name_len": "2048", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_namespace_len": "256", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_object_size": "134217728", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-22T01:14:02.893 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_blocked_by": "16", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_log_entries": "10000", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_pgls": "1024", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_push_cost": "8388608", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_push_objects": "10", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_scrubs": "3", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_trimming_pgs": "2", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_write_op_reply_len": "64", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_max_write_size": "90", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-22T01:14:02.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_profile": "balanced", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-22T01:14:02.895 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_base": "805306368", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_cache_min": "134217728", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target": "4294967296", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target_autotune": "false", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_min_pg_log_entries": "250", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_min_recovery_priority": "0", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-22T01:14:02.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_report_interval": "5", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_num_cache_shards": "32", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_num_op_tracker_shard": "32", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_auto_affinity": "true", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_node": "-1", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_numa_prefer_iface": "true", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objecter_finishers": "1", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore": "bluestore", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore_fuse": "false", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_objectstore_tracing": "false", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_complaint_time": "30.000000", 2024-09-22T01:14:02.897 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_duration": "600", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_size": "20", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_slow_op_size": "20", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_log_threshold": "5", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards": "0", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards_hdd": "5", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_shards_ssd": "8", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_pq_min_cost": "65536", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-22T01:14:02.898 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_queue_cut_off": "high", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_op_thread_timeout": "15", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_open_classes_on_start": "true", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_os_flags": "0", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_peering_op_priority": "255", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_delete_cost": "1048576", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_delete_priority": "5", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_trim_max": "10000", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_log_trim_min": "100", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-22T01:14:02.899 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_crimson": "false", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-22T01:14:02.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_flags": "0", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_min_size": "0", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pg_num": "32", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_pgp_num": "0", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_read_ratio": "70", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_size": "2", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_default_type": "replicated", 2024-09-22T01:14:02.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_push_per_object_cost": "1000", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recover_clone_overlap": "true", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_cost": "20971520", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active": "0", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_max_single_start": "1", 2024-09-22T01:14:02.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_op_priority": "3", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_priority": "5", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep": "0.000000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_repair_during_recovery": "false", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_requested_scrub_priority": "120", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_auto_repair": "false", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-22T01:14:02.903 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_begin_hour": "0", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_begin_week_day": "0", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_chunk_max": "25", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_chunk_min": "5", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_cost": "52428800", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_during_recovery": "false", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_end_hour": "0", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_end_week_day": "0", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_event_cost": "4096", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_invalid_stats": "true", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-22T01:14:02.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_max_preemptions": "5", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_priority": "5", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_scrub_sleep": "0.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_skip_data_digest": "false", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_smart_report_timeout": "5", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_cost": "1048576", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_priority": "5", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-22T01:14:02.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_target_transaction_size": "30", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-22T01:14:02.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_tracing": "false", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_use_stale_snap": "false", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osdc_blkin_trace_all": "false", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_kill_at": "0", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_max_join_drift": "10", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_min": "500", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_min_wait": "0.050000", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_propose_interval": "1.000000", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_max": "500", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_service_trim_min": "250", 2024-09-22T01:14:02.907 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_stash_full_interval": "25", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_trim_max": "500", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "paxos_trim_min": "250", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "perf": "true", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "-", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrv": "", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "public_bind_addr": "-", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "public_network": "", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "public_network_interface": "", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "qat_compressor_busy_polling": "false", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "qat_compressor_enabled": "false", 2024-09-22T01:14:02.908 INFO:teuthology.orchestra.run.smithi096.stdout: "qat_compressor_session_max_number": "256", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_mon_op_timeout": "0", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_osd_op_timeout": "0", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rados_tracing": "false", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_atime_update_interval": "60", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_balance_parent_reads": "false", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_balance_snap_reads": "false", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blkin_trace_all": "false", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache": "true", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-22T01:14:02.909 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_policy": "writearound", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_size": "33554432", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_clone_copy_on_read": "false", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_compression_hint": "none", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_concurrent_management_ops": "10", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_clone_format": "auto", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_data_pool": "", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_features": "61", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_format": "2", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_map_options": "", 2024-09-22T01:14:02.910 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_order": "22", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_pool": "rbd", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_stripe_count": "0", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_default_stripe_unit": "0", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_enable_alloc_hint": "true", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_io_scheduler": "simple", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-22T01:14:02.911 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_order": "24", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_pool": "", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_journal_splay_width": "4", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_localize_parent_reads": "false", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_localize_snap_reads": "false", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-22T01:14:02.912 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-22T01:14:02.913 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_mtime_update_interval": "60", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_non_blocking_aio": "true", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_op_thread_timeout": "60", 2024-09-22T01:14:02.914 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_op_threads": "1", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_parent_cache_enabled": "false", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_plugins": "", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_burst": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_bps_limit": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_exclude_ops": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_burst": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_iops_limit": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-22T01:14:02.915 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_read_from_replica_policy": "default", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-22T01:14:02.916 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_skip_partial_discard": "true", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_tracing": "false", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_validate_names": "true", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_validate_pool": "true", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "restapi_base_url": "", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "restapi_log_level": "", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_acl_grants_max_num": "100", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_admin_entry": "admin", 2024-09-22T01:14:02.917 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_backend_store": "rados", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_barbican_url": "", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_beast_enable_async": "true", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_counters_cache": "false", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-22T01:14:02.918 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_enabled": "true", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_expiry_interval": "900", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cache_lru_size": "25000", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_config_store": "rados", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_content_length_compat": "false", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_copy_obj_progress": "true", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cors_rules_max_num": "100", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_cross_domain_policy": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_addr": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-22T01:14:02.919 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_password": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_kmip_username": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_require_ssl": "true", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-22T01:14:02.920 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_addr": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_auth": "token", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_namespace": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_prefix": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_token_file": "", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_buffersize": "524288", 2024-09-22T01:14:02.921 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_low_speed_time": "300", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_d4n_port": "6379", 2024-09-22T01:14:02.922 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_changes_size": "1000", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_num_shards": "128", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_log_window": "30", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-22T01:14:02.923 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-22T01:14:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dns_name": "", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dns_s3website_name": "", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_dynamic_resharding": "true", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_gc_threads": "true", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_lc_threads": "true", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_ops_log": "false", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_quota_threads": "true", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_static_website": "false", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enable_usage_log": "false", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_enforce_swift_acls": "true", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_exit_timeout_secs": "120", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_expose_bucket": "false", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_extended_http_attrs": "", 2024-09-22T01:14:02.925 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_filter": "none", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_frontends": "beast port=7480", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_deferred": "50", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_objs": "32", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_gc_processor_period": "3600", 2024-09-22T01:14:02.926 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_init_timeout": "300", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_kafka_connection_idle": "300", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-22T01:14:02.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_domain": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_password": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_password_path": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_project": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_tenant": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_token": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_token_path": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_admin_user": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_api_version": "2", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_domain": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_password": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_project": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-22T01:14:02.928 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_barbican_user": "", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_url": "", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_debug_interval": "-1", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_lock_max_time": "90", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_objs": "32", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_rules": "1000", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_worker": "3", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lc_thread_delay": "0", 2024-09-22T01:14:02.929 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_dnattr": "uid", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_searchfilter": "", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ldap_uri": "", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_http_headers": "", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_log_object_name_utc": "false", 2024-09-22T01:14:02.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attr_name_len": "0", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attr_size": "0", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_chunk_size": "4194304", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_listing_results": "1000", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_notify_retries": "10", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_put_param_size": "1048576", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_put_size": "5368709120", 2024-09-22T01:14:02.931 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_max_slo_entries": "1000", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_md_log_max_shards": "64", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_mp_lock_max_time": "600", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-22T01:14:02.932 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_max_gc": "300", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_num_async_rados_threads": "32", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_num_control_oids": "8", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_numa_node": "-1", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-22T01:14:02.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_chunk_size": "100", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_gc_interval": "600", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_thread_timeout": "600", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_op_tracing": "false", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_token": "", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_url": "", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_opa_verify_ssl": "true", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-22T01:14:02.934 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_rados": "false", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_ops_log_socket_path": "", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_push_interval": "2.000000", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_cache_lanes": "3", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_cache_partitions": "3", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_print_continue": "true", 2024-09-22T01:14:02.935 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rados_tracing": "false", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm": "", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm_id": "", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_region": "", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-22T01:14:02.936 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_request_uri": "", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_batch_size": "64", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_max_aio": "128", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_num_logs": "16", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_reshard_thread_interval": "600", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_resolve_cname": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_run_sync_thread": "true", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-22T01:14:02.937 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_scheduler_type": "throttler", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_script_uri": "", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_service_provider_name": "", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_client_id": "", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_client_secret": "", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_entry": "sts", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_key": "sts", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_min_session_duration": "900", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sts_token_introspection_url": "", 2024-09-22T01:14:02.938 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_account_in_url": "false", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_auth_entry": "auth", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_auth_url": "", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_custom_header": "", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_need_stats": "true", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_tenant_name": "", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_token_expiration": "86400", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_url": "", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_url_prefix": "swift", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_lease_period": "120", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-22T01:14:02.939 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_thread_pool_size": "512", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_comment": "", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_createby": "", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_encoding": "", 2024-09-22T01:14:02.940 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_flag": "false", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_origin": "", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_torrent_tracker": "", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_trust_forwarded_https": "false", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_max_shards": "32", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_usage_max_user_shards": "1", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_use_opa_authz": "false", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_counters_cache": "false", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-22T01:14:02.941 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_max_buckets": "1000", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_policies_max_num": "100", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_user_unique_email": "true", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_verify_ssl": "true", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone": "", 2024-09-22T01:14:02.942 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone_id": "", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup": "", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup_id": "", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgwlc_auto_session_clear": "true", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_block_size": "4096", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_size": "536870912", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-22T01:14:02.943 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_index_type": "binary_search", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_partition_filters": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_perf": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-22T01:14:02.944 INFO:teuthology.orchestra.run.smithi096.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "run_dir": "/var/run/ceph", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_block_create": "true", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_cache_lru_size": "67108864", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_default_max_object_size": "16777216", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_device_size": "53687091200", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_full_integrity_check": "false", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_capacity": "16", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_main_device_type": "SSD", 2024-09-22T01:14:02.945 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "seastore_segment_size": "67108864", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "set_keepcaps": "false", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "setgroup": "", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "setuser": "", 2024-09-22T01:14:02.946 INFO:teuthology.orchestra.run.smithi096.stdout: "setuser_match_path": "", 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "thp": "false", 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "threadpool_default_timeout": "60", 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout: "throttler_perf_counter": "true" 2024-09-22T01:14:02.947 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:02.949 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-22T01:14:03.090 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a: {} 2024-09-22T01:14:03.090 INFO:teuthology.orchestra.run.smithi096.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:14:03.103 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b: {} 2024-09-22T01:14:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:14:03.111 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c: {} 2024-09-22T01:14:03.112 INFO:teuthology.orchestra.run.smithi096.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:14:03.127 DEBUG:teuthology.orchestra.run.smithi096:> type ceph-objectstore-tool 2024-09-22T01:14:03.131 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T01:14:03.131 DEBUG:teuthology.orchestra.run.smithi150:> type ceph-objectstore-tool 2024-09-22T01:14:03.137 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-22T01:14:03.137 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-22T01:14:03.149 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-22T01:14:03.150 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-22T01:14:03.150 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:03.150 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:03.150 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:03.150 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-22T01:14:03.151 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-22T01:14:03.151 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-22T01:14:03.151 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:03.151 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-22T01:14:03.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:03.152 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-22T01:14:03.152 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:14:03.152 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-22T01:14:03.180 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:03.179+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:03.204 INFO:teuthology.orchestra.run.smithi096.stdout:mapped writes DISABLED 2024-09-22T01:14:03.204 INFO:teuthology.orchestra.run.smithi096.stdout:mapped reads DISABLED 2024-09-22T01:14:03.204 INFO:teuthology.orchestra.run.smithi096.stdout:Seed set to 2103 2024-09-22T01:14:03.253 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:03.253+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:03.296 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:03.297 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:03.311 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:03.318 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:03.341 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:03.341 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:03.354 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:03.351+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:03.371 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:03.371 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:03.402 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:03.427 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:03.427 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:03.435 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:03.435 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:03.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:03.455 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:03.453+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:03.469 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:03.469 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:03.501 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:03.501 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:03.521 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:03.522 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:03.573 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-22T01:14:03.571+0000 7f33f656f640 -1 mon.a@0(leader).osd e23 definitely_dead 0 2024-09-22T01:14:03.573 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:03.571+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:03.577 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:03.590 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:03.675 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:03.675+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:03.744 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:03.744 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:03.757 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:03.759 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:03.757+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:03.860 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:03.861+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:03.918 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:03.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:03.975 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:03.975+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:04.045 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.046 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.058 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.3. 2024-09-22T01:14:04.092 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:14:04.095 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:04.091+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:04.167 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:04.167+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:04.200 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.201 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:04.262 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:04.261+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:04.363 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:04.361+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.371 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:04.464 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:04.463+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.507 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:04.567 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:04.567+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:04.666 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:04.663+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:04.683 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.684 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:04.767 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:04.765+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:04.860 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.861 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:04.867 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:04.865+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:04.874 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:04.968 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:04.967+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:04.996 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:04.997 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:05.011 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:05.070 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:05.069+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:05.144 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:05.144 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:05.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:05.204 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:05.205+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:05.272 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:05.273+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:05.328 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:05.328 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:05.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:05.384 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:05.385+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:05.473 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:05.473+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:05.510 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:05.510 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:05.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:05.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:05.612 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:05.613+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:05.675 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:05.675+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:05.687 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:05.687 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:05.693 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:05.694 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:05.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:05.711 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:05.712 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:05.730 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:05.731 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:05.747 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:05.748 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:05.767 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:05.767 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:05.785 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:05.785 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:05.787 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:05.789+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:05.805 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:05.805 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:05.824 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:05.824 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:05.871 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:05.871 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:05.871 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:05.878 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:05.877+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:05.885 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:05.979 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:05.979+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:06.047 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.047 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:06.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:105 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'67, mlcod=18'64)", 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.507786+0000", 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.538652771000001, 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044549480000000002, 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507786+0000", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507786+0000", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507787+0000", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507802+0000", 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5223e-05 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507866+0000", 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3565999999999994e-05 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.507892+0000", 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6222999999999999e-05 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508463+0000", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00057058200000000003 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508518+0000", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5059999999999998e-05 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512172+0000", 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0036537480000000001 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512241+0000", 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9356000000000005e-05 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:106 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'68, mlcod=18'64)", 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.508383+0000", 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.538055913000001, 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058072879999999999, 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.052 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508383+0000", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508383+0000", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508384+0000", 2024-09-22T01:14:06.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6300000000000004e-07 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508467+0000", 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3252999999999997e-05 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508498+0000", 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0830000000000001e-05 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.508507+0000", 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3559999999999999e-06 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509232+0000", 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072482500000000001 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509284+0000", 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2272999999999999e-05 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514121+0000", 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048368090000000001 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514190+0000", 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9078999999999994e-05 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:107 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'69, mlcod=18'64)", 2024-09-22T01:14:06.056 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.509179+0000", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.537259532, 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050547470000000001, 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509179+0000", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509179+0000", 2024-09-22T01:14:06.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509181+0000", 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.75e-06 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509196+0000", 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4966e-05 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509222+0000", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6573000000000001e-05 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509239+0000", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.662e-05 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510656+0000", 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001416736 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510690+0000", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4162999999999998e-05 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514195+0000", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003505139 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514234+0000", 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8800000000000001e-05 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.060 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:108 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'70, mlcod=18'64)", 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.509605+0000", 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.536833239, 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0070011589999999999, 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509605+0000", 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509605+0000", 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509607+0000", 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.373e-06 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509622+0000", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509669+0000", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7193e-05 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.509692+0000", 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3424000000000001e-05 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511471+0000", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017789360000000001 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511511+0000", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9712999999999997e-05 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516528+0000", 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0050174140000000004 2024-09-22T01:14:06.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516607+0000", 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8202999999999996e-05 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:109 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'71, mlcod=18'64)", 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.510108+0000", 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.536330705999999, 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080218040000000004, 2024-09-22T01:14:06.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510108+0000", 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510108+0000", 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510109+0000", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.483e-06 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510192+0000", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2355999999999996e-05 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510220+0000", 2024-09-22T01:14:06.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7733e-05 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510249+0000", 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9286999999999999e-05 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512329+0000", 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020805229999999999 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512367+0000", 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7886999999999998e-05 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518053+0000", 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056856789999999999 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518130+0000", 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6855999999999997e-05 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:110 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'72, mlcod=18'64)", 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.510827+0000", 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.535611362000001, 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0073550730000000002, 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510827+0000", 2024-09-22T01:14:06.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510827+0000", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510829+0000", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3370000000000001e-06 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510839+0000", 2024-09-22T01:14:06.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0606e-05 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510868+0000", 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9064000000000002e-05 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.510888+0000", 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9792999999999998e-05 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513554+0000", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0026657859999999999 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513588+0000", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4513e-05 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518136+0000", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004547171 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518182+0000", 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6802999999999999e-05 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:111 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'73, mlcod=18'64)", 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.511253+0000", 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.535185295, 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0069952520000000004, 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.074 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511253+0000", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511253+0000", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511255+0000", 2024-09-22T01:14:06.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2699999999999999e-06 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511266+0000", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1297e-05 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511298+0000", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2112e-05 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511332+0000", 2024-09-22T01:14:06.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4412999999999997e-05 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514346+0000", 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030134509999999999 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514399+0000", 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3205000000000003e-05 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518196+0000", 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037963480000000002 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518249+0000", 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3155999999999997e-05 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:112 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'74, mlcod=18'65)", 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.511993+0000", 2024-09-22T01:14:06.078 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.534445351, 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0072728999999999997, 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511993+0000", 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511993+0000", 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.511995+0000", 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5999999999999999e-06 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512054+0000", 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9009999999999999e-05 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512104+0000", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512118+0000", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.415e-05 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.515139+0000", 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030213369999999998 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.515175+0000", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5871999999999998e-05 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.519206+0000", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040306159999999999 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.519266+0000", 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0375999999999999e-05 2024-09-22T01:14:06.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:113 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'75, mlcod=18'65)", 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.512440+0000", 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.533998964, 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068805689999999996, 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512440+0000", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512440+0000", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512441+0000", 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2529999999999999e-06 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512452+0000", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362999999999999e-05 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512476+0000", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3366e-05 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512496+0000", 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0106999999999999e-05 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516070+0000", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003574139 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516109+0000", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8692999999999997e-05 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.519269+0000", 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031602319999999998 2024-09-22T01:14:06.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.519320+0000", 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1415999999999997e-05 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:114 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'65)", 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.512860+0000", 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.533578371000001, 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075547310000000003, 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512860+0000", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512860+0000", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512862+0000", 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.333e-06 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512874+0000", 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2466999999999999e-05 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512904+0000", 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0306000000000001e-05 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.512921+0000", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6657e-05 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516699+0000", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037779269999999999 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.516739+0000", 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9709999999999998e-05 2024-09-22T01:14:06.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.520326+0000", 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035870780000000001 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.520415+0000", 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9252999999999994e-05 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:115 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'77, mlcod=18'66)", 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.513367+0000", 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.533071888, 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096750840000000005, 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513367+0000", 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513367+0000", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513369+0000", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.793e-06 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513455+0000", 2024-09-22T01:14:06.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5986e-05 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513482+0000", 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.709e-05 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.513499+0000", 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7059e-05 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.517255+0000", 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037557580000000001 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.517286+0000", 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1733000000000003e-05 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.522947+0000", 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0056604560000000003 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.523042+0000", 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5209000000000003e-05 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:116 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'78, mlcod=18'66)", 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.514147+0000", 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.532291990999999, 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0089552739999999992, 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514147+0000", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514147+0000", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514148+0000", 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0899999999999999e-06 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514157+0000", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514173+0000", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6263e-05 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514184+0000", 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1263e-05 2024-09-22T01:14:06.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518336+0000", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041514250000000003 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518371+0000", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4938999999999999e-05 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.523047+0000", 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046762069999999999 2024-09-22T01:14:06.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.523102+0000", 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5037000000000002e-05 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:117 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'79, mlcod=18'66)", 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.514706+0000", 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.531733159, 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084472909999999991, 2024-09-22T01:14:06.100 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514706+0000", 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514706+0000", 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514707+0000", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.297e-06 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514718+0000", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0852999999999999e-05 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514750+0000", 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.188e-05 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.514797+0000", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7453000000000001e-05 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518896+0000", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040994209999999998 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.518928+0000", 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.188e-05 2024-09-22T01:14:06.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.523104+0000", 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0041754110000000004 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.523153+0000", 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9095999999999999e-05 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:134 1.0 e18/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:51.984045+0000", 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.062394185, 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003743868, 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984045+0000", 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984045+0000", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984045+0000", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9599999999999999e-07 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984138+0000", 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2997000000000004e-05 2024-09-22T01:14:06.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984147+0000", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5210000000000002e-06 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984151+0000", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5480000000000003e-06 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984183+0000", 2024-09-22T01:14:06.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1418999999999999e-05 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.984197+0000", 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3886000000000001e-05 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.987742+0000", 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035453780000000001 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:51.987788+0000", 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6422999999999999e-05 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:154 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'108, mlcod=19'105)", 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.721939+0000", 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.32450019, 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042699269999999998, 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.109 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721939+0000", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721939+0000", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721940+0000", 2024-09-22T01:14:06.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4130000000000001e-06 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721951+0000", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0637e-05 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721969+0000", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8397e-05 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.721997+0000", 2024-09-22T01:14:06.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7606e-05 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722342+0000", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034576399999999998 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722392+0000", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9262999999999999e-05 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.726143+0000", 2024-09-22T01:14:06.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037514269999999999 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.726208+0000", 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5419999999999994e-05 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:155 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'109, mlcod=19'105)", 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.722501+0000", 2024-09-22T01:14:06.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.323938001, 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058906410000000003, 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722501+0000", 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722501+0000", 2024-09-22T01:14:06.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722502+0000", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722659+0000", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000157059 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722690+0000", 2024-09-22T01:14:06.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0919e-05 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.722707+0000", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7456999999999999e-05 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723077+0000", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036979 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723132+0000", 2024-09-22T01:14:06.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5287000000000001e-05 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728328+0000", 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051958000000000004 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728391+0000", 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3418999999999997e-05 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:156 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'110, mlcod=19'105)", 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.723185+0000", 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.32325359, 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052507530000000004, 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723185+0000", 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723185+0000", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723186+0000", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1000000000000001e-06 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723199+0000", 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2933e-05 2024-09-22T01:14:06.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723217+0000", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.755e-05 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723230+0000", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3114e-05 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724734+0000", 2024-09-22T01:14:06.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001504635 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724786+0000", 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1493000000000003e-05 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728397+0000", 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0036108949999999998 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728436+0000", 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9032999999999997e-05 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:157 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'111, mlcod=19'105)", 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.723732+0000", 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.322706217, 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0047525600000000003, 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723732+0000", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723732+0000", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723734+0000", 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.906e-06 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723756+0000", 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1267000000000002e-05 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723788+0000", 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2573000000000001e-05 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.723810+0000", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1950000000000002e-05 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725480+0000", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001670164 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725532+0000", 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1242000000000002e-05 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728446+0000", 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029146380000000002 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728485+0000", 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8819999999999998e-05 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:158 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'112, mlcod=19'105)", 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.724375+0000", 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.322064032, 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004148751, 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724375+0000", 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724375+0000", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724377+0000", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724486+0000", 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108995 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724514+0000", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.834e-05 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.724530+0000", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5889999999999999e-05 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.726299+0000", 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001769386 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.726371+0000", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1379999999999998e-05 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728488+0000", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021173070000000001 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.728523+0000", 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5593000000000003e-05 2024-09-22T01:14:06.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4207.0:159 1.0 e19/17 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'113, mlcod=19'105)", 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:52.725224+0000", 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.321214715, 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0040379020000000003, 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725224+0000", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725224+0000", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725226+0000", 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6530000000000001e-06 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725258+0000", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1906000000000003e-05 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725282+0000", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4243000000000002e-05 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.725316+0000", 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3893999999999998e-05 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.727738+0000", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024219039999999999 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.727769+0000", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0993e-05 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.729167+0000", 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001398076 2024-09-22T01:14:06.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:52.729262+0000", 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5233000000000001e-05 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.135 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:14:06.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:06.137 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:06.137+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:06.237 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:06.237+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:06.306 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.307 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:06.307 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:06.307 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:06.364 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:06.365+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:06.439 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:06.439+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:06.486 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.487 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:06.487 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:06.487 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:06.487 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.500 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:06.503 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:14:06.543 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:06.545+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:06.626 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4220.0:5 2.2 e22/20 2:5cea7035:::rbd_info:head v 22'1, mlcod=0'0)", 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:56.024162+0000", 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.60479782, 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001877809, 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.627 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024162+0000", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024162+0000", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024163+0000", 2024-09-22T01:14:06.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5700000000000001e-07 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024183+0000", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.022e-05 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024219+0000", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6115999999999998e-05 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024236+0000", 2024-09-22T01:14:06.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6993000000000001e-05 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024374+0000", 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013797899999999999 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.024424+0000", 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0296000000000001e-05 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.025924+0000", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0014993090000000001 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:56.026040+0000", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116039 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4220.0:7 2.2 e23/20 2:5cea7035:::rbd_info:head v 23'2, mlcod=0'0)", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:13:57.048451+0000", 2024-09-22T01:14:06.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.580508236, 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00216103, 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048451+0000", 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048451+0000", 2024-09-22T01:14:06.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048452+0000", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8999999999999996e-07 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048460+0000", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4470000000000006e-06 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048487+0000", 2024-09-22T01:14:06.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6659999999999999e-05 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048502+0000", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5362999999999998e-05 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048643+0000", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014047800000000001 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.048690+0000", 2024-09-22T01:14:06.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6783000000000003e-05 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.050536+0000", 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0018463329999999999 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:13:57.050612+0000", 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6376000000000002e-05 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:06.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:06.636 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:06.641 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:06.641+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:06.741 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:06.739+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:06.807 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.807 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:06.807 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:06.807 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:06.868 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:06.869+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:06.939 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:06.939 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:06.939 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:06.939 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:06.939 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:06.942 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:06.941+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:06.954 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:07.044 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:07.045+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:07.117 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:07.117 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:07.118 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [] 2024-09-22T01:14:07.118 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:07.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:07.179 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:07.179+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:07.244 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:07.243+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:07.252 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:07.252 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:07.252 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:07.252 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:07.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:07.345 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:07.345+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:07.424 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:07.425 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:07.425 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:07.437 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:07.483 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:07.483+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:07.547 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:07.545+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:07.551 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:07.551 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.0342839500000001, 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091594199999999999, 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:07.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9700000000000003e-07 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518473+0000", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.302e-06 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518496+0000", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2456999999999999e-05 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518505+0000", 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:14:07.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518582+0000", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7409000000000004e-05 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518609+0000", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6906999999999998e-05 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.519311+0000", 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070160600000000002 2024-09-22T01:14:07.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.519383+0000", 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1907000000000001e-05 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:13 3.e e27/24 3:77402a29:::rbd_object_map.11242152224b:head v 27'1, mlcod=0'0)", 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:06.529002+0000", 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.023748165, 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00082124300000000004, 2024-09-22T01:14:07.556 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529002+0000", 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529002+0000", 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529003+0000", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6299999999999999e-07 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529007+0000", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529019+0000", 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1556000000000001e-05 2024-09-22T01:14:07.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529025+0000", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4649999999999999e-06 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529097+0000", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1279000000000007e-05 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529115+0000", 2024-09-22T01:14:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8586000000000001e-05 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529776+0000", 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000661033 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.529824+0000", 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7474e-05 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:07.561 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:07.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:07.648 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:07.649+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:07.730 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:07.730 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:07.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:07.748 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:07.749+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:07.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:07.859 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:07.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:07.860 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:07.860 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:07.860 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:07.860 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:07.859+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:07.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:07.959 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:07.959+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:07.980 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:07.980 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.044903334000001, 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:08.031 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:08.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:08.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:08.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.794670375999999, 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011381442, 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234702+0000", 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234713+0000", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0507e-05 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234735+0000", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1800000000000001e-05 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234748+0000", 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3583e-05 2024-09-22T01:14:08.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243350+0000", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086011609999999995 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243483+0000", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308499999999999 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243702+0000", 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021935499999999999 2024-09-22T01:14:08.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245600+0000", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018977219999999999 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245927+0000", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032715799999999997 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246017+0000", 2024-09-22T01:14:08.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9752000000000004e-05 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246082+0000", 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5813000000000002e-05 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.79459409, 2024-09-22T01:14:08.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013805090000000001, 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234779+0000", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-06 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234789+0000", 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.042e-05 2024-09-22T01:14:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234808+0000", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9256e-05 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234826+0000", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7825999999999998e-05 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244277+0000", 2024-09-22T01:14:08.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094506409999999992 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244411+0000", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134513 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244661+0000", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024963399999999999 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245702+0000", 2024-09-22T01:14:08.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001040736 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248290+0000", 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025886429999999998 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248450+0000", 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015967800000000001 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248582+0000", 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132316 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.522530120999999, 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011449948999999999, 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506842+0000", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506918+0000", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5715000000000003e-05 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506925+0000", 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4599999999999997e-06 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506929+0000", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2520000000000001e-06 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510441+0000", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035115350000000001 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510484+0000", 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2024-09-22T01:14:08.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510560+0000", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6184000000000001e-05 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513176+0000", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026151540000000002 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518234+0000", 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050580019999999998 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518267+0000", 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3800999999999997e-05 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518291+0000", 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3855000000000001e-05 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.522433118999999, 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011448767, 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:08.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506939+0000", 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506942+0000", 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8420000000000001e-06 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506945+0000", 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9349999999999996e-06 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506949+0000", 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511010+0000", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040611680000000004 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511054+0000", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4255999999999998e-05 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511140+0000", 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5841999999999996e-05 2024-09-22T01:14:08.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513889+0000", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027490869999999999 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518324+0000", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044343489999999998 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518358+0000", 2024-09-22T01:14:08.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4737999999999998e-05 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518387+0000", 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8572999999999999e-05 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.522360051, 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446914, 2024-09-22T01:14:08.060 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507012+0000", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507015+0000", 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:14:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507020+0000", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507023+0000", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511408+0000", 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043849700000000002 2024-09-22T01:14:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511451+0000", 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2735999999999999e-05 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511539+0000", 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7766e-05 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513894+0000", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023553100000000002 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519395+0000", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055011189999999996 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519431+0000", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5457999999999998e-05 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519458+0000", 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7682e-05 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.522157273000001, 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015085121999999999, 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507215+0000", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507301+0000", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6791000000000003e-05 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507310+0000", 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.653e-06 2024-09-22T01:14:08.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507323+0000", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2648e-05 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511896+0000", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045732380000000003 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511954+0000", 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8319000000000002e-05 2024-09-22T01:14:08.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512028+0000", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513897+0000", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001869587 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522235+0000", 2024-09-22T01:14:08.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083377590000000001 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522264+0000", 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9431999999999999e-05 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522299+0000", 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4860999999999997e-05 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.521993342000002, 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01505534, 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507379+0000", 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:14:08.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507382+0000", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.258e-06 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507388+0000", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.263e-06 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507392+0000", 2024-09-22T01:14:08.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.771e-06 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512533+0000", 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051413939999999997 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512618+0000", 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4338000000000005e-05 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512742+0000", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012398199999999999 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513903+0000", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001161147 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522355+0000", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084519360000000002 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522399+0000", 2024-09-22T01:14:08.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4508000000000001e-05 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522433+0000", 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4177000000000002e-05 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.521911227, 2024-09-22T01:14:08.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015058092, 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507461+0000", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507463+0000", 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2024-09-22T01:14:08.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507475+0000", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.146e-05 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507479+0000", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1049999999999997e-06 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513231+0000", 2024-09-22T01:14:08.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057521810000000003 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513296+0000", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4764999999999994e-05 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513388+0000", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1847000000000003e-05 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.516279+0000", 2024-09-22T01:14:08.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028914119999999999 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522465+0000", 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006186352 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522496+0000", 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0988000000000002e-05 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522518+0000", 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1792000000000001e-05 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054690301000001, 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01429044, 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 131 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974682+0000", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974824+0000", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014190599999999999 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974839+0000", 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974849+0000", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8369999999999992e-06 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977235+0000", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023857219999999998 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977380+0000", 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014566199999999999 2024-09-22T01:14:08.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977640+0000", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025971799999999998 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.980903+0000", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032634040000000001 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988830+0000", 2024-09-22T01:14:08.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260430000000007 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988904+0000", 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4016000000000004e-05 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988971+0000", 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7929000000000006e-05 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.088 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054502492000001, 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014424996000000001, 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974870+0000", 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974876+0000", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974889+0000", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2670000000000001e-05 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974896+0000", 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9370000000000004e-06 2024-09-22T01:14:08.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979031+0000", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041348749999999997 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979163+0000", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132142 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979423+0000", 2024-09-22T01:14:08.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025945100000000001 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981009+0000", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001585801 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989179+0000", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081698940000000005 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989243+0000", 2024-09-22T01:14:08.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4480000000000006e-05 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989294+0000", 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0742999999999999e-05 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:08.094 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054426472999999, 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484576000000001, 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 133 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974946+0000", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0499999999999999e-06 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974952+0000", 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-22T01:14:08.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974964+0000", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2164000000000001e-05 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974976+0000", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1875999999999999e-05 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981087+0000", 2024-09-22T01:14:08.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061111660000000003 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981224+0000", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013626900000000001 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981437+0000", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021382899999999999 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985221+0000", 2024-09-22T01:14:08.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037832999999999999 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989306+0000", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004085506 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989379+0000", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2909000000000003e-05 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989429+0000", 2024-09-22T01:14:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0377000000000002e-05 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054323236999998, 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018165239999999999, 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 134 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975049+0000", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975179+0000", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129722 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975197+0000", 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8113000000000001e-05 2024-09-22T01:14:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975207+0000", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5100000000000004e-06 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982219+0000", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070121760000000002 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982352+0000", 2024-09-22T01:14:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133249 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982617+0000", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026466800000000002 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989070+0000", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064531299999999996 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.992931+0000", 2024-09-22T01:14:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003860774 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993092+0000", 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016092499999999999 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993213+0000", 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121836 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054146965000001, 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018294939, 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 135 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975225+0000", 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975231+0000", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7699999999999998e-06 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975243+0000", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1644000000000001e-05 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975250+0000", 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.503e-06 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983862+0000", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086120459999999999 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983989+0000", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012705299999999999 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.984185+0000", 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019606499999999999 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989159+0000", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049732120000000003 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993238+0000", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040793449999999998 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993372+0000", 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013405200000000001 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993519+0000", 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733899999999999 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054105732, 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018936457, 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 136 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8400000000000004e-07 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975272+0000", 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975284+0000", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2393e-05 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975292+0000", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.763e-06 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985342+0000", 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010050026 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985551+0000", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208755 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985988+0000", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043766699999999998 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989513+0000", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003525233 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994036+0000", 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045223279999999999 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994117+0000", 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1549000000000001e-05 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994202+0000", 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4735999999999996e-05 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.054049895999999, 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019072927, 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975323+0000", 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:14:08.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975483+0000", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160416 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975500+0000", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7127000000000001e-05 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975509+0000", 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4399999999999994e-06 2024-09-22T01:14:08.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987086+0000", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011576606999999999 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987414+0000", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032788700000000001 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987806+0000", 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039176100000000002 2024-09-22T01:14:08.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993673+0000", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058668970000000003 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994227+0000", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055439900000000002 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994330+0000", 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010278 2024-09-22T01:14:08.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994394+0000", 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4542999999999995e-05 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.053844414, 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019163698999999999, 2024-09-22T01:14:08.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975528+0000", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.017e-06 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975534+0000", 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975547+0000", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2806000000000001e-05 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975554+0000", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989583+0000", 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014028175 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989829+0000", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024677100000000001 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.990235+0000", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040547400000000001 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994418+0000", 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041833579999999999 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994534+0000", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011541500000000001 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994623+0000", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9703000000000005e-05 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994691+0000", 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7366000000000005e-05 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.053804763999999, 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021194425999999999, 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-22T01:14:08.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975577+0000", 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4669999999999993e-06 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975587+0000", 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975594+0000", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1770000000000001e-06 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991392+0000", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015798764 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991632+0000", 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023988199999999999 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991978+0000", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034544099999999999 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996441+0000", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004463608 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996541+0000", 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9623000000000005e-05 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996644+0000", 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103416 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996761+0000", 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116532 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.133 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.053765974000001, 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022802374, 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:08.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975606+0000", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0299999999999998e-07 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975744+0000", 2024-09-22T01:14:08.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013788599999999999 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975767+0000", 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3003e-05 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975781+0000", 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.383e-05 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994785+0000", 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019003995999999999 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994930+0000", 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144683 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.995124+0000", 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019457099999999999 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998089+0000", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029647440000000001 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998221+0000", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178199999999999 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998301+0000", 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9839999999999995e-05 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998408+0000", 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107236 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.053574571999999, 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024814201000000001, 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:08.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:08.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975798+0000", 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975807+0000", 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.533e-06 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:08.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975817+0000", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9599999999999995e-06 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975824+0000", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3289999999999996e-06 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996882+0000", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021058038000000001 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:08.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997106+0000", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022396800000000001 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997464+0000", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035745699999999999 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998789+0000", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001325384 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:08.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000443+0000", 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001653493 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000521+0000", 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000611+0000", 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9662999999999999e-05 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:08.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:08.145 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:08.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:08.147 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:08.143+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:08.201 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:08.202 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:08.219 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:08.219 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:08.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:08.238 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:08.246 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:08.243+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:08.259 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:08.259 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:08.280 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:08.280 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:08.300 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:08.301 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:08.315 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:08.316 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:08.322 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:08.322 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:08.331 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:08.347 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:08.345+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:08.437 INFO:teuthology.orchestra.run.smithi096.stdout:1 write 0x86858fd thru 0x86861ef (0x8f3 bytes) 2024-09-22T01:14:08.447 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:08.447+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:08.460 INFO:teuthology.orchestra.run.smithi096.stdout:2 writesame 0xdf6c284 thru 0xdf7c137 data_size 0x2(0xfeb4 bytes) 2024-09-22T01:14:08.460 INFO:teuthology.orchestra.run.smithi096.stdout:3 write 0x37522c5 thru 0x375a8a3 (0x85df bytes) 2024-09-22T01:14:08.465 INFO:teuthology.orchestra.run.smithi096.stdout:7 write 0xb7d6e26 thru 0xb7dc191 (0x536c bytes) 2024-09-22T01:14:08.472 INFO:teuthology.orchestra.run.smithi096.stdout:11 punch from 0xa1c7ef4 to 0xa1d2751, (0xa85d bytes) 2024-09-22T01:14:08.472 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xc794f12 2024-09-22T01:14:08.474 INFO:teuthology.orchestra.run.smithi096.stdout:13 trunc from 0xdf7c138 to 0xc794f12 2024-09-22T01:14:08.502 INFO:teuthology.orchestra.run.smithi096.stdout:14 write 0xe22efc6 thru 0xe23e71e (0xf759 bytes) 2024-09-22T01:14:08.502 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:08.502 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:08.507 INFO:teuthology.orchestra.run.smithi096.stdout:15 read 0xa5ede2b thru 0xa5f93ad (0xb583 bytes) 2024-09-22T01:14:08.507 INFO:teuthology.orchestra.run.smithi096.stdout:16 punch from 0x7947276 to 0x794b906, (0x4690 bytes) 2024-09-22T01:14:08.507 INFO:teuthology.orchestra.run.smithi096.stdout:17 read 0x5a1ca88 thru 0x5a21811 (0x4d8a bytes) 2024-09-22T01:14:08.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:08.520 INFO:teuthology.orchestra.run.smithi096.stdout:18 write 0xe67c815 thru 0xe681a01 (0x51ed bytes) 2024-09-22T01:14:08.526 INFO:teuthology.orchestra.run.smithi096.stdout:19 read 0xb629bd3 thru 0xb639407 (0xf835 bytes) 2024-09-22T01:14:08.527 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:14:08.527 INFO:teuthology.orchestra.run.smithi096.stdout:20 write 0x347640f thru 0x3482e62 (0xca54 bytes) 2024-09-22T01:14:08.530 INFO:teuthology.orchestra.run.smithi096.stdout:21 write 0x93771f2 thru 0x937fe12 (0x8c21 bytes) 2024-09-22T01:14:08.535 INFO:teuthology.orchestra.run.smithi096.stdout:22 read 0x5da87b9 thru 0x5dae64b (0x5e93 bytes) 2024-09-22T01:14:08.535 INFO:teuthology.orchestra.run.smithi096.stdout:25 writesame 0xbdc933 thru 0xbe8346 data_size 0x2(0xba14 bytes) 2024-09-22T01:14:08.549 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:08.549+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:08.556 INFO:teuthology.orchestra.run.smithi096.stdout:28 punch from 0xe0f628 to 0xe127f2, (0x31ca bytes) 2024-09-22T01:14:08.556 INFO:teuthology.orchestra.run.smithi096.stdout:29 write 0xc2579f3 thru 0xc2650f0 (0xd6fe bytes) 2024-09-22T01:14:08.564 INFO:teuthology.orchestra.run.smithi096.stdout:30 write 0xcc8850e thru 0xcc90202 (0x7cf5 bytes) 2024-09-22T01:14:08.577 INFO:teuthology.orchestra.run.smithi096.stdout:31 write 0xeb4a123 thru 0xeb4b67c (0x155a bytes) 2024-09-22T01:14:08.582 INFO:teuthology.orchestra.run.smithi096.stdout:32 write 0xc198369 thru 0xc19c67f (0x4317 bytes) 2024-09-22T01:14:08.585 INFO:teuthology.orchestra.run.smithi096.stdout:33 read 0x59c8a90 thru 0x59cedbc (0x632d bytes) 2024-09-22T01:14:08.585 INFO:teuthology.orchestra.run.smithi096.stdout:34 read 0xe9475c2 thru 0xe9506dc (0x911b bytes) 2024-09-22T01:14:08.587 INFO:teuthology.orchestra.run.smithi096.stdout:35 read 0xb0c7c2d thru 0xb0d68dd (0xecb1 bytes) 2024-09-22T01:14:08.587 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.587 INFO:teuthology.orchestra.run.smithi096.stdout:38 write 0x346418b thru 0x3473e12 (0xfc88 bytes) 2024-09-22T01:14:08.590 INFO:teuthology.orchestra.run.smithi096.stdout:39 read 0xbef3b6b thru 0xbef73ab (0x3841 bytes) 2024-09-22T01:14:08.590 INFO:teuthology.orchestra.run.smithi096.stdout:41 write 0x3f41220 thru 0x3f4d813 (0xc5f4 bytes) 2024-09-22T01:14:08.595 INFO:teuthology.orchestra.run.smithi096.stdout:42 writesame 0xceb5ccf thru 0xcec2b6a data_size 0xe(0xce9c bytes) 2024-09-22T01:14:08.600 INFO:teuthology.orchestra.run.smithi096.stdout:43 punch from 0x1d3a2b8 to 0x1d41571, (0x72b9 bytes) 2024-09-22T01:14:08.605 INFO:teuthology.orchestra.run.smithi096.stdout:44 trunc from 0xeb4b67d to 0xae2ea49 2024-09-22T01:14:08.624 INFO:teuthology.orchestra.run.smithi096.stdout:45 write 0x282b764 thru 0x28330d8 (0x7975 bytes) 2024-09-22T01:14:08.627 INFO:teuthology.orchestra.run.smithi096.stdout:46 write 0x5be29ff thru 0x5bede70 (0xb472 bytes) 2024-09-22T01:14:08.631 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.631 INFO:teuthology.orchestra.run.smithi096.stdout:49 read 0x3b908e0 thru 0x3b9a218 (0x9939 bytes) 2024-09-22T01:14:08.631 INFO:teuthology.orchestra.run.smithi096.stdout:53 writesame 0x342f615 thru 0x34335c0 data_size 0x19(0x3fac bytes) 2024-09-22T01:14:08.634 INFO:teuthology.orchestra.run.smithi096.stdout:54 punch from 0xa5cae5 to 0xa6b875, (0xed90 bytes) 2024-09-22T01:14:08.634 INFO:teuthology.orchestra.run.smithi096.stdout:55 punch from 0x5b8aae to 0x5bc521, (0x3a73 bytes) 2024-09-22T01:14:08.645 INFO:teuthology.orchestra.run.smithi096.stdout:56 writesame 0xbc00836 thru 0xbc05b14 data_size 0x1(0x52df bytes) 2024-09-22T01:14:08.645 INFO:teuthology.orchestra.run.smithi096.stdout:57 writesame 0x106fc29 thru 0x107d6b5 data_size 0x1(0xda8d bytes) 2024-09-22T01:14:08.646 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:08.646 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:08.650 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:08.649+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:08.661 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:08.688 INFO:teuthology.orchestra.run.smithi096.stdout:58 write 0xa09fa1f thru 0xa0a5cad (0x628f bytes) 2024-09-22T01:14:08.696 INFO:teuthology.orchestra.run.smithi096.stdout:59 writesame 0x4f67524 thru 0x4f719b7 data_size 0x6(0xa494 bytes) 2024-09-22T01:14:08.705 INFO:teuthology.orchestra.run.smithi096.stdout:60 writesame 0x11be319 thru 0x11c0c39 data_size 0x1(0x2921 bytes) 2024-09-22T01:14:08.714 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.714 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.714 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.725 INFO:teuthology.orchestra.run.smithi096.stdout:64 write 0xc6a9b91 thru 0xc6ada16 (0x3e86 bytes) 2024-09-22T01:14:08.732 INFO:teuthology.orchestra.run.smithi096.stdout:66 write 0x985ca2d thru 0x985cb35 (0x109 bytes) 2024-09-22T01:14:08.736 INFO:teuthology.orchestra.run.smithi096.stdout:67 read 0x39ed7bd thru 0x39f7399 (0x9bdd bytes) 2024-09-22T01:14:08.736 INFO:teuthology.orchestra.run.smithi096.stdout:68 punch from 0xb18b425 to 0xb190874, (0x544f bytes) 2024-09-22T01:14:08.737 INFO:teuthology.orchestra.run.smithi096.stdout:70 punch from 0x4444066 to 0x444f1c1, (0xb15b bytes) 2024-09-22T01:14:08.737 INFO:teuthology.orchestra.run.smithi096.stdout:71 punch from 0x2d26acd to 0x2d2d827, (0x6d5a bytes) 2024-09-22T01:14:08.737 INFO:teuthology.orchestra.run.smithi096.stdout:73 writesame 0x89ac74a thru 0x89b667a data_size 0x1(0x9f31 bytes) 2024-09-22T01:14:08.750 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:08.747+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:08.777 INFO:teuthology.orchestra.run.smithi096.stdout:74 trunc from 0xc6ada17 to 0x1a7cb80 2024-09-22T01:14:08.806 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.811 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:08.812 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:08.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:76 writesame 0xbda25b7 thru 0xbda4c21 data_size 0x7(0x266b bytes) 2024-09-22T01:14:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:81 read 0x89dd5d9 thru 0x89ed2bc (0xfce4 bytes) 2024-09-22T01:14:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:84 punch from 0xbe5458 to 0xbe767f, (0x2227 bytes) 2024-09-22T01:14:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:85 punch from 0x9886b1a to 0x98899f9, (0x2edf bytes) 2024-09-22T01:14:08.831 INFO:teuthology.orchestra.run.smithi096.stdout:89 punch from 0x9c59c7d to 0x9c66469, (0xc7ec bytes) 2024-09-22T01:14:08.843 INFO:teuthology.orchestra.run.smithi096.stdout:90 write 0xe292526 thru 0xe297a77 (0x5552 bytes) 2024-09-22T01:14:08.847 INFO:teuthology.orchestra.run.smithi096.stdout:91 write 0x8927d69 thru 0x8931967 (0x9bff bytes) 2024-09-22T01:14:08.851 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:08.853+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:08.853 INFO:teuthology.orchestra.run.smithi096.stdout:92 read 0xc9942f3 thru 0xc99b599 (0x72a7 bytes) 2024-09-22T01:14:08.853 INFO:teuthology.orchestra.run.smithi096.stdout:93 writesame 0x3de3b26 thru 0x3ded37e data_size 0x1(0x9859 bytes) 2024-09-22T01:14:08.875 INFO:teuthology.orchestra.run.smithi096.stdout:94 write 0xc8256c8 thru 0xc82f960 (0xa299 bytes) 2024-09-22T01:14:08.882 INFO:teuthology.orchestra.run.smithi096.stdout:95 read 0x3a7d979 thru 0x3a88c24 (0xb2ac bytes) 2024-09-22T01:14:08.882 INFO:teuthology.orchestra.run.smithi096.stdout:96 read 0xd21ab1a thru 0xd2273aa (0xc891 bytes) 2024-09-22T01:14:08.882 INFO:teuthology.orchestra.run.smithi096.stdout:97 punch from 0x547c4ae to 0x5488183, (0xbcd5 bytes) 2024-09-22T01:14:08.882 INFO:teuthology.orchestra.run.smithi096.stdout:99 write 0xb41adbb thru 0xb42874b (0xd991 bytes) 2024-09-22T01:14:08.888 INFO:teuthology.orchestra.run.smithi096.stdout:101 read 0xb132c23 thru 0xb13a0bb (0x7499 bytes) 2024-09-22T01:14:08.888 INFO:teuthology.orchestra.run.smithi096.stdout:102 writesame 0x5475a2a thru 0x547ef1a data_size 0xd(0x94f1 bytes) 2024-09-22T01:14:08.895 INFO:teuthology.orchestra.run.smithi096.stdout:103 punch from 0xb86e8fa to 0xb87c9c9, (0xe0cf bytes) 2024-09-22T01:14:08.895 INFO:teuthology.orchestra.run.smithi096.stdout:104 writesame 0x416d750 thru 0x4175fbe data_size 0x1(0x886f bytes) 2024-09-22T01:14:08.915 INFO:teuthology.orchestra.run.smithi096.stdout:105 writesame 0xb126fc7 thru 0xb13457c data_size 0x5(0xd5b6 bytes) 2024-09-22T01:14:08.928 INFO:teuthology.orchestra.run.smithi096.stdout:106 read 0x82b2893 thru 0x82bb4ff (0x8c6d bytes) 2024-09-22T01:14:08.928 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:08.928 INFO:teuthology.orchestra.run.smithi096.stdout:108 read 0x92dc5ad thru 0x92e8f40 (0xc994 bytes) 2024-09-22T01:14:08.928 INFO:teuthology.orchestra.run.smithi096.stdout:109 punch from 0xca1b9c0 to 0xca1bff6, (0x636 bytes) 2024-09-22T01:14:08.937 INFO:teuthology.orchestra.run.smithi096.stdout:110 trunc from 0xe297a78 to 0x6b0d089 2024-09-22T01:14:08.952 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:08.953+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:08.960 INFO:teuthology.orchestra.run.smithi096.stdout:111 trunc from 0x6b0d089 to 0x5adc334 2024-09-22T01:14:08.961 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:08.961 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:08.971 INFO:teuthology.orchestra.run.smithi096.stdout:112 read 0x1b7a75b thru 0x1b88e1f (0xe6c5 bytes) 2024-09-22T01:14:08.972 INFO:teuthology.orchestra.run.smithi096.stdout:113 write 0x33c25f5 thru 0x33cb112 (0x8b1e bytes) 2024-09-22T01:14:08.976 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:08.979 INFO:teuthology.orchestra.run.smithi096.stdout:115 read 0x16767c0 thru 0x168392f (0xd170 bytes) 2024-09-22T01:14:08.980 INFO:teuthology.orchestra.run.smithi096.stdout:116 writesame 0x4cb6f1a thru 0x4cc5c13 data_size 0x3(0xecfa bytes) 2024-09-22T01:14:09.004 INFO:teuthology.orchestra.run.smithi096.stdout:117 write 0x332dd3b thru 0x33305dc (0x28a2 bytes) 2024-09-22T01:14:09.006 INFO:teuthology.orchestra.run.smithi096.stdout:118 read 0x5013da6 thru 0x5021788 (0xd9e3 bytes) 2024-09-22T01:14:09.006 INFO:teuthology.orchestra.run.smithi096.stdout:119 read 0x31d8b70 thru 0x31ddbce (0x505f bytes) 2024-09-22T01:14:09.009 INFO:teuthology.orchestra.run.smithi096.stdout:120 trunc from 0x5adc334 to 0x553f2f3 2024-09-22T01:14:09.025 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.025 INFO:teuthology.orchestra.run.smithi096.stdout:124 read 0x389ec9e thru 0x38a7ccc (0x902f bytes) 2024-09-22T01:14:09.026 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.026 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.048 INFO:teuthology.orchestra.run.smithi096.stdout:128 write 0xea33644 thru 0xea37838 (0x41f5 bytes) 2024-09-22T01:14:09.053 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xccf8b74 2024-09-22T01:14:09.053 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:09.051+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:09.055 INFO:teuthology.orchestra.run.smithi096.stdout:130 trunc from 0xea37839 to 0xccf8b74 2024-09-22T01:14:09.082 INFO:teuthology.orchestra.run.smithi096.stdout:132 trunc from 0xccf8b74 to 0x429d6c9 2024-09-22T01:14:09.099 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.099 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.102 INFO:teuthology.orchestra.run.smithi096.stdout:133 punch from 0x2f7cf20 to 0x2f822f4, (0x53d4 bytes) 2024-09-22T01:14:09.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:09.118 INFO:teuthology.orchestra.run.smithi096.stdout:134 writesame 0x81dbe90 thru 0x81ea6df data_size 0x54(0xe850 bytes) 2024-09-22T01:14:09.120 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:09.120 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:09.120 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-22T01:14:09.120 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-22T01:14:09.133 INFO:teuthology.orchestra.run.smithi096.stdout:135 writesame 0xb262475 thru 0xb2662f4 data_size 0x28(0x3e80 bytes) 2024-09-22T01:14:09.142 INFO:teuthology.orchestra.run.smithi096.stdout:136 writesame 0xc1ca050 thru 0xc1d8145 data_size 0x41(0xe0f6 bytes) 2024-09-22T01:14:09.142 INFO:teuthology.orchestra.run.smithi096.stdout:137 punch from 0x19df9cc to 0x19ec8ac, (0xcee0 bytes) 2024-09-22T01:14:09.142 INFO:teuthology.orchestra.run.smithi096.stdout:138 write 0x81e01d thru 0x82df59 (0xff3d bytes) 2024-09-22T01:14:09.145 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.154 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:09.151+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:09.154 INFO:teuthology.orchestra.run.smithi096.stdout:140 write 0xd59d0ff thru 0xd5a148a (0x438c bytes) 2024-09-22T01:14:09.158 INFO:teuthology.orchestra.run.smithi096.stdout:141 read 0x73e3c1b thru 0x73f138e (0xd774 bytes) 2024-09-22T01:14:09.159 INFO:teuthology.orchestra.run.smithi096.stdout:142 write 0x373cffa thru 0x37481fa (0xb201 bytes) 2024-09-22T01:14:09.166 INFO:teuthology.orchestra.run.smithi096.stdout:143 trunc from 0xd5a148b to 0xbab8922 2024-09-22T01:14:09.180 INFO:teuthology.orchestra.run.smithi096.stdout:144 read 0x347352b thru 0x34824b8 (0xef8e bytes) 2024-09-22T01:14:09.181 INFO:teuthology.orchestra.run.smithi096.stdout:145 read 0x6871a11 thru 0x687bd13 (0xa303 bytes) 2024-09-22T01:14:09.196 INFO:teuthology.orchestra.run.smithi096.stdout:146 trunc from 0xbab8922 to 0xf9fcc8 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:147 read 0x706549 thru 0x7148df (0xe397 bytes) 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:149 punch from 0x9f1435 to 0x9f2b34, (0x16ff bytes) 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:150 read 0x586c2e thru 0x588e48 (0x221b bytes) 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:152 read 0xe726c5 thru 0xe7f1c7 (0xcb03 bytes) 2024-09-22T01:14:09.214 INFO:teuthology.orchestra.run.smithi096.stdout:154 trunc from 0xf9fcc8 to 0x1100d5c 2024-09-22T01:14:09.223 INFO:teuthology.orchestra.run.smithi096.stdout:155 read 0xeb1bc0 thru 0xeb6d84 (0x51c5 bytes) 2024-09-22T01:14:09.223 INFO:teuthology.orchestra.run.smithi096.stdout:156 read 0x5a2aab thru 0x5b0c79 (0xe1cf bytes) 2024-09-22T01:14:09.223 INFO:teuthology.orchestra.run.smithi096.stdout:157 read 0x934c31 thru 0x93d9ea (0x8dba bytes) 2024-09-22T01:14:09.238 INFO:teuthology.orchestra.run.smithi096.stdout:158 write 0x40fd905 thru 0x4101ade (0x41da bytes) 2024-09-22T01:14:09.254 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:09.251+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:09.261 INFO:teuthology.orchestra.run.smithi096.stdout:159 write 0xbc3a626 thru 0xbc3d9c4 (0x339f bytes) 2024-09-22T01:14:09.263 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.263 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.265 INFO:teuthology.orchestra.run.smithi096.stdout:160 write 0x57b79a3 thru 0x57c0892 (0x8ef0 bytes) 2024-09-22T01:14:09.270 INFO:teuthology.orchestra.run.smithi096.stdout:162 read 0x869a949 thru 0x86a73c8 (0xca80 bytes) 2024-09-22T01:14:09.270 INFO:teuthology.orchestra.run.smithi096.stdout:163 read 0x4244fb0 thru 0x4254882 (0xf8d3 bytes) 2024-09-22T01:14:09.271 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:09.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:09.288 INFO:teuthology.orchestra.run.smithi096.stdout:168 trunc from 0xbc3d9c5 to 0x6c8f13 2024-09-22T01:14:09.310 INFO:teuthology.orchestra.run.smithi096.stdout:169 writesame 0x36da3ef thru 0x36e698a data_size 0x2(0xc59c bytes) 2024-09-22T01:14:09.310 INFO:teuthology.orchestra.run.smithi096.stdout:171 clone 1 order 23 su 32768 sc 13 2024-09-22T01:14:09.355 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:09.355+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:09.402 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.402 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:09.451 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-22T01:14:09.451+0000 7f33f656f640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-22T01:14:09.463 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:09.465+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:14:09.519 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.3. 2024-09-22T01:14:09.540 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.540 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.554 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-22T01:14:09.556 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:09.603 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:09.603+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:09.657 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:09.657+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:09.725 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.725 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:09.758 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:09.755+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:09.859 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:09.859+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:09.906 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:09.906 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:09.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:09.960 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:09.959+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:10.041 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:10.041 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:10.055 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:10.100 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:10.101+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:10.161 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:10.159+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:10.172 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:10.172 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:10.186 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:10.263 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:10.263+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:10.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:10.344 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:10.345 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:10.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:10.387 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:10.387+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:10.464 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:10.465+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:10.485 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:10.485 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:10.499 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:10.500 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:10.500 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:10.517 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:10.517 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:10.540 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:10.540 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:10.561 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:10.561 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:10.582 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:10.583 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:10.595 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:10.597+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:10.602 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:10.602 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:10.621 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:10.622 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:10.640 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:10.640 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:10.665 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:10.665+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:10.674 INFO:teuthology.orchestra.run.smithi096.stdout:172 write 0x435b79b thru 0x4363273 (0x7ad9 bytes) 2024-09-22T01:14:10.679 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:10.680 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:10.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:10.706 INFO:teuthology.orchestra.run.smithi096.stdout:173 write 0xed8a1c3 thru 0xed8a499 (0x2d7 bytes) 2024-09-22T01:14:10.709 INFO:teuthology.orchestra.run.smithi096.stdout:174 punch from 0x9cb417d to 0x9cb9c37, (0x5aba bytes) 2024-09-22T01:14:10.710 INFO:teuthology.orchestra.run.smithi096.stdout:175 writesame 0x25a878c thru 0x25b59b4 data_size 0xb(0xd229 bytes) 2024-09-22T01:14:10.715 INFO:teuthology.orchestra.run.smithi096.stdout:176 write 0x1711750 thru 0x17167f3 (0x50a4 bytes) 2024-09-22T01:14:10.717 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.717 INFO:teuthology.orchestra.run.smithi096.stdout:179 read 0x4c5c4d8 thru 0x4c6b46b (0xef94 bytes) 2024-09-22T01:14:10.717 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.717 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.718 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.718 INFO:teuthology.orchestra.run.smithi096.stdout:187 writesame 0xd9a504 thru 0xda9a15 data_size 0x26(0xf512 bytes) 2024-09-22T01:14:10.727 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.727 INFO:teuthology.orchestra.run.smithi096.stdout:189 punch from 0x6deae59 to 0x6df2677, (0x781e bytes) 2024-09-22T01:14:10.727 INFO:teuthology.orchestra.run.smithi096.stdout:191 write 0xd016e1 thru 0xd0e568 (0xce88 bytes) 2024-09-22T01:14:10.736 INFO:teuthology.orchestra.run.smithi096.stdout:192 write 0x9d602eb thru 0x9d623f0 (0x2106 bytes) 2024-09-22T01:14:10.740 INFO:teuthology.orchestra.run.smithi096.stdout:193 read 0x3830d9 thru 0x38d8c8 (0xa7f0 bytes) 2024-09-22T01:14:10.740 INFO:teuthology.orchestra.run.smithi096.stdout:194 read 0x9f39c0d thru 0x9f40990 (0x6d84 bytes) 2024-09-22T01:14:10.740 INFO:teuthology.orchestra.run.smithi096.stdout:196 write 0xbf531b thru 0xc01514 (0xc1fa bytes) 2024-09-22T01:14:10.745 INFO:teuthology.orchestra.run.smithi096.stdout:198 punch from 0x3f29d85 to 0x3f2aedf, (0x115a bytes) 2024-09-22T01:14:10.745 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.745 INFO:teuthology.orchestra.run.smithi096.stdout:200 read 0x81b922 thru 0x821272 (0x5951 bytes) 2024-09-22T01:14:10.745 INFO:teuthology.orchestra.run.smithi096.stdout:202 writesame 0xaebe56a thru 0xaec0907 data_size 0x2(0x239e bytes) 2024-09-22T01:14:10.751 INFO:teuthology.orchestra.run.smithi096.stdout:204 punch from 0x531421 to 0x531a80, (0x65f bytes) 2024-09-22T01:14:10.751 INFO:teuthology.orchestra.run.smithi096.stdout:208 writesame 0x8a2097d thru 0x8a27cf7 data_size 0x25(0x737b bytes) 2024-09-22T01:14:10.755 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.755 INFO:teuthology.orchestra.run.smithi096.stdout:211 read 0x9c127a thru 0x9d0477 (0xf1fe bytes) 2024-09-22T01:14:10.756 INFO:teuthology.orchestra.run.smithi096.stdout:212 punch from 0xaf8dea0 to 0xaf94312, (0x6472 bytes) 2024-09-22T01:14:10.756 INFO:teuthology.orchestra.run.smithi096.stdout:213 write 0xb86c43b thru 0xb872c72 (0x6838 bytes) 2024-09-22T01:14:10.761 INFO:teuthology.orchestra.run.smithi096.stdout:214 write 0x52d6dfb thru 0x52e2eaa (0xc0b0 bytes) 2024-09-22T01:14:10.766 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:10.763+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:10.769 INFO:teuthology.orchestra.run.smithi096.stdout:215 punch from 0xe526d0e to 0xe528b92, (0x1e84 bytes) 2024-09-22T01:14:10.769 INFO:teuthology.orchestra.run.smithi096.stdout:216 read 0xa6879c0 thru 0xa68fe2d (0x846e bytes) 2024-09-22T01:14:10.769 INFO:teuthology.orchestra.run.smithi096.stdout:217 write 0xbb68c57 thru 0xbb6e354 (0x56fe bytes) 2024-09-22T01:14:10.774 INFO:teuthology.orchestra.run.smithi096.stdout:218 write 0xb19fc0d thru 0xb1a76ed (0x7ae1 bytes) 2024-09-22T01:14:10.781 INFO:teuthology.orchestra.run.smithi096.stdout:219 read 0x31aa388 thru 0x31b0842 (0x64bb bytes) 2024-09-22T01:14:10.781 INFO:teuthology.orchestra.run.smithi096.stdout:220 punch from 0x7f10b6 to 0x7f6977, (0x58c1 bytes) 2024-09-22T01:14:10.781 INFO:teuthology.orchestra.run.smithi096.stdout:221 read 0x8a3029e thru 0x8a37c72 (0x79d5 bytes) 2024-09-22T01:14:10.782 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.782 INFO:teuthology.orchestra.run.smithi096.stdout:223 writesame 0x37ef6a2 thru 0x37f6997 data_size 0x36(0x72f6 bytes) 2024-09-22T01:14:10.785 INFO:teuthology.orchestra.run.smithi096.stdout:224 writesame 0x7ad402d thru 0x7adc900 data_size 0x15(0x88d4 bytes) 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:225 punch from 0x8714ff9 to 0x872181f, (0xc826 bytes) 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:227 read 0xbf3c300 thru 0xbf412b4 (0x4fb5 bytes) 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:231 read 0x5b40dee thru 0x5b42362 (0x1575 bytes) 2024-09-22T01:14:10.789 INFO:teuthology.orchestra.run.smithi096.stdout:233 punch from 0x4d40b5b to 0x4d4f8c4, (0xed69 bytes) 2024-09-22T01:14:10.790 INFO:teuthology.orchestra.run.smithi096.stdout:234 punch from 0xa5a1293 to 0xa5b1225, (0xff92 bytes) 2024-09-22T01:14:10.790 INFO:teuthology.orchestra.run.smithi096.stdout:236 read 0x5d52a1f thru 0x5d58e97 (0x6479 bytes) 2024-09-22T01:14:10.790 INFO:teuthology.orchestra.run.smithi096.stdout:237 write 0xde6ef0f thru 0xde7bfee (0xd0e0 bytes) 2024-09-22T01:14:10.796 INFO:teuthology.orchestra.run.smithi096.stdout:238 writesame 0x9171883 thru 0x9171db3 data_size 0x1(0x531 bytes) 2024-09-22T01:14:10.807 INFO:teuthology.orchestra.run.smithi096.stdout:239 trunc from 0xed8a49a to 0xbd051a8 2024-09-22T01:14:10.830 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:10.830 INFO:teuthology.orchestra.run.smithi096.stdout:241 read 0xa425e5b thru 0xa42d2b8 (0x745e bytes) 2024-09-22T01:14:10.840 INFO:teuthology.orchestra.run.smithi096.stdout:242 trunc from 0xbd051a8 to 0x855fb72 2024-09-22T01:14:10.858 INFO:teuthology.orchestra.run.smithi096.stdout:243 punch from 0x69fc40e to 0x6a0307a, (0x6c6c bytes) 2024-09-22T01:14:10.858 INFO:teuthology.orchestra.run.smithi096.stdout:244 clone 2 order 25 su 131072 sc 11 2024-09-22T01:14:10.865 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:10.865 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:10.865 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:10.865 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:10.865 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:10.867 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:10.867+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:10.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:10.967 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:10.967+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:559 3.b 3:d2499b78:::rbd_object_map.11242152224b.0000000000000003:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.662501+0000", 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.34056032800000002, 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064095799999999996, 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 559 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662501+0000", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662501+0000", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662504+0000", 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5170000000000002e-06 2024-09-22T01:14:11.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662518+0000", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4796e-05 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662552+0000", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3312999999999998e-05 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662568+0000", 2024-09-22T01:14:11.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6702999999999999e-05 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662696+0000", 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012804900000000001 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.662856+0000", 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000159729 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.663142+0000", 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000285851 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:604 3.c e29/24 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:head v 29'1, mlcod=0'0)", 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.737948+0000", 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.26511345800000002, 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012654020000000001, 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.737948+0000", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.737948+0000", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.737949+0000", 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0399999999999995e-07 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.738043+0000", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4358999999999996e-05 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.738075+0000", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1810000000000002e-05 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.738092+0000", 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7343e-05 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.738170+0000", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7847000000000002e-05 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.738191+0000", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0591999999999999e-05 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.739168+0000", 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0009776120000000001 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.739213+0000", 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5135000000000001e-05 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:606 3.c e29/24 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head v 29'2, mlcod=0'0)", 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.741573+0000", 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.26148872099999998, 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001147117, 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741573+0000", 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741573+0000", 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741573+0000", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741585+0000", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1776e-05 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741596+0000", 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1127e-05 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741603+0000", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7309999999999996e-06 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741660+0000", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6907000000000003e-05 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.741679+0000", 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.742683+0000", 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001003713 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.742720+0000", 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6761000000000003e-05 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:617 3.0 e29/24 3:0088113c:::rbd_data.1124a8bc99f2.000000000000000f:head v 29'6, mlcod=27'4)", 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.753049+0000", 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.25001262400000002, 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010887799999999999, 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753049+0000", 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753049+0000", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753049+0000", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753061+0000", 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1918e-05 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753120+0000", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9354000000000001e-05 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753137+0000", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6212999999999999e-05 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753225+0000", 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8827999999999997e-05 2024-09-22T01:14:11.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.753258+0000", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2796999999999998e-05 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.754097+0000", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00083826500000000004 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.754138+0000", 2024-09-22T01:14:11.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1010999999999997e-05 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:632 3.0 e29/24 3:0088113c:::rbd_data.1124a8bc99f2.000000000000000f:head v 29'7, mlcod=27'5)", 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.777868+0000", 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.225193372, 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001098573, 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777868+0000", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777868+0000", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777870+0000", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1900000000000002e-06 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777888+0000", 2024-09-22T01:14:11.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8090000000000001e-05 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777931+0000", 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2376000000000002e-05 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.777952+0000", 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0959e-05 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.778050+0000", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8720000000000003e-05 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.778077+0000", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6591e-05 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.778922+0000", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084496500000000004 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.778967+0000", 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4681999999999997e-05 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:639 3.c e29/24 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head v 29'3, mlcod=29'1)", 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.790227+0000", 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.21283449099999999, 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001121031, 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790227+0000", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790227+0000", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790228+0000", 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.421e-06 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790264+0000", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6012e-05 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790283+0000", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8572000000000001e-05 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790296+0000", 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3149999999999999e-05 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790354+0000", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7930999999999998e-05 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.790372+0000", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7778000000000002e-05 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.791271+0000", 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00089893500000000001 2024-09-22T01:14:11.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.791348+0000", 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7231999999999995e-05 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:661 3.c e29/24 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head v 29'4, mlcod=29'2)", 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.822231+0000", 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.18083081200000001, 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001811177, 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822231+0000", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822231+0000", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822232+0000", 2024-09-22T01:14:11.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.42e-06 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822243+0000", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0693000000000001e-05 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822268+0000", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5043999999999999e-05 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822279+0000", 2024-09-22T01:14:11.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822438+0000", 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015946600000000001 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822571+0000", 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013258899999999999 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.823957+0000", 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013860719999999999 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.824042+0000", 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5127e-05 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:658 3.0 e29/24 3:0088113c:::rbd_data.1124a8bc99f2.000000000000000f:head v 29'8, mlcod=29'6)", 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.822265+0000", 2024-09-22T01:14:11.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.180796172, 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001238264, 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822265+0000", 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822265+0000", 2024-09-22T01:14:11.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822267+0000", 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.81e-06 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822313+0000", 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5946999999999999e-05 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822346+0000", 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3458999999999999e-05 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822367+0000", 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0905999999999999e-05 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822507+0000", 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013959399999999999 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.822533+0000", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5840999999999999e-05 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.823383+0000", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084988200000000005 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.823504+0000", 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120825 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:676 3.0 e29/24 3:0088113c:::rbd_data.1124a8bc99f2.000000000000000f:head v 29'9, mlcod=29'7)", 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.15870140199999999, 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0032775980000000001, 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844362+0000", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7770000000000001e-06 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844375+0000", 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:14:11.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844406+0000", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269000000000002e-05 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844419+0000", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.331e-05 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844501+0000", 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1818999999999997e-05 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844529+0000", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408e-05 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.847523+0000", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029937520000000001 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.847638+0000", 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011443299999999999 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:679 3.c e29/24 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head v 29'5, mlcod=29'3)", 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.158200234, 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033864889999999999, 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844862+0000", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2099999999999995e-07 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844870+0000", 2024-09-22T01:14:11.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8860000000000004e-06 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844882+0000", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1917e-05 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844890+0000", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5210000000000002e-06 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844955+0000", 2024-09-22T01:14:11.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4814999999999995e-05 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844979+0000", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3442e-05 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.848169+0000", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003190657 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.848248+0000", 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8330000000000004e-05 2024-09-22T01:14:11.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:11.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:11.045 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:11.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:11.091 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:11.091+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:11.169 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:11.167+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:11.179 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:11.179 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:11.179 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:11.179 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:11.193 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:11.270 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:11.269+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:11.358 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:11.358 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:11.358 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:11.358 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:11.358 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:11.371 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:11.369+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:11.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:11.472 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:11.471+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9776389559999998, 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091594199999999999, 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518467+0000", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9700000000000003e-07 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518473+0000", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.302e-06 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518496+0000", 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2456999999999999e-05 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518505+0000", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0569999999999999e-06 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518582+0000", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7409000000000004e-05 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.518609+0000", 2024-09-22T01:14:11.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6906999999999998e-05 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.519311+0000", 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070160600000000002 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:06.519383+0000", 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1907000000000001e-05 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:497 3.1c e29/24 3:3831950c:::rbd_directory:head v 29'5, mlcod=27'3)", 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.541893+0000", 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.95421206599999997, 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040689630000000001, 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541893+0000", 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541893+0000", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541894+0000", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541898+0000", 2024-09-22T01:14:11.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2239999999999997e-06 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541908+0000", 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7280000000000007e-06 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541917+0000", 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4099999999999997e-06 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.541986+0000", 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8885000000000002e-05 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.542009+0000", 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3133e-05 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.545913+0000", 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039041840000000002 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.545962+0000", 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9193000000000001e-05 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:598 3.1d e29/28 3:b99e8344:::rbd_data.1124a8bc99f2.0000000000000006:head v 29'4, mlcod=0'0)", 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.726987+0000", 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76911865300000004, 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010973700000000001, 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.726987+0000", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.726987+0000", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.726988+0000", 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.181e-06 2024-09-22T01:14:11.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.727011+0000", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2826000000000001e-05 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.727028+0000", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6713000000000001e-05 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.727038+0000", 2024-09-22T01:14:11.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9119999999999993e-06 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.727110+0000", 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2614999999999995e-05 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.727131+0000", 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1135e-05 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.728038+0000", 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00090624900000000003 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.728084+0000", 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6739000000000001e-05 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:604 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1048576~25961] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.736168+0000", 2024-09-22T01:14:11.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.75993785199999997, 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00212692, 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 604 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736168+0000", 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736168+0000", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736169+0000", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5260000000000001e-06 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736184+0000", 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:14:11.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736198+0000", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-05 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736204+0000", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6259999999999997e-06 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736262+0000", 2024-09-22T01:14:11.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8019999999999997e-05 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736395+0000", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132411 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.736547+0000", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151724 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.737411+0000", 2024-09-22T01:14:11.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086421899999999995 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.738222+0000", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081099500000000001 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.738263+0000", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0797999999999998e-05 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.738295+0000", 2024-09-22T01:14:11.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1951e-05 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:606 3.c 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head [set-alloc-hint object_size 8388608 write_size 8388608,write 4293355~8454] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.740068+0000", 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.75603741999999996, 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017528909999999999, 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 606 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740068+0000", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740068+0000", 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740069+0000", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5199999999999995e-07 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740080+0000", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0936e-05 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740092+0000", 2024-09-22T01:14:11.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2459999999999999e-05 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740099+0000", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2580000000000001e-06 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740142+0000", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3708999999999998e-05 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740238+0000", 2024-09-22T01:14:11.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5297000000000006e-05 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.740323+0000", 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5279999999999997e-05 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.741719+0000", 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00139638 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.741738+0000", 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8280000000000001e-05 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.741770+0000", 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2316e-05 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.741821+0000", 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1023000000000003e-05 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:624 3.13 e29/28 3:ce2ff269:::rbd_data.1124a8bc99f2.0000000000000019:head v 29'1, mlcod=0'0)", 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.761441+0000", 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73466484600000004, 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00096642500000000003, 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761441+0000", 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761441+0000", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761442+0000", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7549999999999999e-06 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761489+0000", 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6975999999999999e-05 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761515+0000", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5568999999999999e-05 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761527+0000", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2214e-05 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761593+0000", 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5455000000000003e-05 2024-09-22T01:14:11.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.761615+0000", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1945000000000001e-05 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.762370+0000", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075512300000000001 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.762407+0000", 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7388000000000002e-05 2024-09-22T01:14:11.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:631 3.13 e29/28 3:ce7f988b:::rbd_data.1124a8bc99f2.0000000000000011:head v 29'2, mlcod=0'0)", 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.774675+0000", 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72143012200000001, 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000902873, 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774675+0000", 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774675+0000", 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774676+0000", 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0640000000000001e-06 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774703+0000", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6957e-05 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774718+0000", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4891e-05 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774725+0000", 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0790000000000003e-06 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774782+0000", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6898000000000003e-05 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.774803+0000", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0492e-05 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.775543+0000", 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073997099999999997 2024-09-22T01:14:11.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.775578+0000", 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5521000000000001e-05 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:638 3.13 e29/28 3:ce7f988b:::rbd_data.1124a8bc99f2.0000000000000011:head v 29'3, mlcod=29'1)", 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.788464+0000", 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.70764124799999994, 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010741119999999999, 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788464+0000", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788464+0000", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788465+0000", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.094e-06 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788498+0000", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2487000000000003e-05 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788519+0000", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-05 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788532+0000", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3390999999999999e-05 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788627+0000", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4433999999999997e-05 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788653+0000", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6548000000000001e-05 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.789505+0000", 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085221899999999998 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.789538+0000", 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2909e-05 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:639 3.c 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head [write 1507328~18689] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.788506+0000", 2024-09-22T01:14:11.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.70759956899999998, 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020190849999999999, 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 639 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788506+0000", 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788506+0000", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788508+0000", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8169999999999999e-06 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788550+0000", 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2283999999999997e-05 2024-09-22T01:14:11.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788593+0000", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2904000000000001e-05 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788603+0000", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0207000000000001e-05 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788678+0000", 2024-09-22T01:14:11.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4410000000000001e-05 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788756+0000", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8084999999999995e-05 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.788898+0000", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141862 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.789918+0000", 2024-09-22T01:14:11.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010206390000000001 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.790418+0000", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049947900000000005 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.790462+0000", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4564000000000002e-05 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.790525+0000", 2024-09-22T01:14:11.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2834000000000002e-05 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:648 3.f 3:f38133eb:::rbd_data.1124a8bc99f2.0000000000000022:head [set-alloc-hint object_size 8388608 write_size 8388608,write 1146880~16367] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.796015+0000", 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.70009040700000003, 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022005480000000001, 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 648 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796015+0000", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796015+0000", 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796017+0000", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.424e-06 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796031+0000", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4226999999999999e-05 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796051+0000", 2024-09-22T01:14:11.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0169e-05 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796062+0000", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1127999999999999e-05 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796163+0000", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101282 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796370+0000", 2024-09-22T01:14:11.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020675199999999999 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796502+0000", 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000131643 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.798133+0000", 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001630821 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.798144+0000", 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1198e-05 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.798177+0000", 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3306000000000003e-05 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.798216+0000", 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8597999999999999e-05 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:646 3.13 e29/28 3:cb01149b:::rbd_data.1124a8bc99f2.0000000000000020:head v 29'4, mlcod=29'2)", 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.796063+0000", 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.70004253999999999, 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001374861, 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796063+0000", 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796063+0000", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796064+0000", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7100000000000011e-07 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796093+0000", 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8836999999999999e-05 2024-09-22T01:14:11.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796108+0000", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5631e-05 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796115+0000", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796178+0000", 2024-09-22T01:14:11.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3155000000000001e-05 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.796194+0000", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5979999999999999e-05 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.797331+0000", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011365699999999999 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.797438+0000", 2024-09-22T01:14:11.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107181 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:653 3.f 3:f38133eb:::rbd_data.1124a8bc99f2.0000000000000022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.814205+0000", 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68190036499999995, 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003110341, 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 653 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814205+0000", 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814205+0000", 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814208+0000", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.875e-06 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814223+0000", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.466e-05 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814263+0000", 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0682000000000001e-05 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814293+0000", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9649999999999999e-05 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814463+0000", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016994199999999999 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814715+0000", 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025210100000000002 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.815037+0000", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032174100000000001 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817051+0000", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020145419999999998 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817173+0000", 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122009 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817248+0000", 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4834999999999998e-05 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817315+0000", 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7304000000000005e-05 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:651 3.13 e29/28 3:cb01149b:::rbd_data.1124a8bc99f2.0000000000000020:head v 29'5, mlcod=29'3)", 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.814776+0000", 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68132936600000005, 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013329640000000001, 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814776+0000", 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814776+0000", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814778+0000", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3599999999999999e-06 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814789+0000", 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1657999999999999e-05 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814807+0000", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7479999999999999e-05 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814817+0000", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9329999999999996e-06 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814935+0000", 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011870300000000001 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814963+0000", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7486000000000001e-05 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.816049+0000", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001086425 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.816109+0000", 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9919e-05 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.561 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:654 3.1c e29/24 3:3ce09b50:::rbd_data.1124a8bc99f2.0000000000000023:head v 29'7, mlcod=29'5)", 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.814922+0000", 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.68118399299999999, 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022707029999999998, 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814922+0000", 2024-09-22T01:14:11.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814922+0000", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814924+0000", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814941+0000", 2024-09-22T01:14:11.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7025999999999999e-05 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814971+0000", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0667999999999998e-05 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.814990+0000", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8865e-05 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.815122+0000", 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178699999999999 2024-09-22T01:14:11.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.815243+0000", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012126 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817092+0000", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001848761 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.817192+0000", 2024-09-22T01:14:11.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100253 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:661 3.c 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head [truncate 6848512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.820522+0000", 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.67558349200000001, 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002795657, 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 661 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820522+0000", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820522+0000", 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820523+0000", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2720000000000001e-06 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820529+0000", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9030000000000001e-06 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820546+0000", 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6347000000000001e-05 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820593+0000", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7892999999999997e-05 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820674+0000", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0346000000000006e-05 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820783+0000", 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000108992 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.820950+0000", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016699999999999999 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821641+0000", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00069119699999999995 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823175+0000", 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001534203 2024-09-22T01:14:11.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823247+0000", 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2211000000000004e-05 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823318+0000", 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0292999999999999e-05 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:660 3.13 e29/28 3:ce7f988b:::rbd_data.1124a8bc99f2.0000000000000011:head v 29'6, mlcod=29'4)", 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.821460+0000", 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.67464544000000004, 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00178538, 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821460+0000", 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821460+0000", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821462+0000", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821478+0000", 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5296000000000001e-05 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821508+0000", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0837000000000003e-05 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821526+0000", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7972999999999999e-05 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821653+0000", 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000126749 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.821704+0000", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0936999999999997e-05 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823140+0000", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014357490000000001 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823245+0000", 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105659 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:662 3.13 e29/28 3:ce2ff269:::rbd_data.1124a8bc99f2.0000000000000019:head v 29'7, mlcod=29'4)", 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.822552+0000", 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.67355373200000002, 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010866000000000001, 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822552+0000", 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822552+0000", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822554+0000", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1090000000000001e-06 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822569+0000", 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5542000000000001e-05 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822600+0000", 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0973999999999999e-05 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822619+0000", 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8326e-05 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822704+0000", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5679000000000004e-05 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.822728+0000", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3587e-05 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823520+0000", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00079175600000000001 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.823638+0000", 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000118627 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:679 3.c 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head [truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.842731+0000", 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.65337404499999996, 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0047310609999999999, 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 679 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842731+0000", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842731+0000", 2024-09-22T01:14:11.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842734+0000", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3269999999999999e-06 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842743+0000", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6399999999999992e-06 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842774+0000", 2024-09-22T01:14:11.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0796000000000001e-05 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842791+0000", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6430999999999999e-05 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.842932+0000", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014165200000000001 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843072+0000", 2024-09-22T01:14:11.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000139963 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843392+0000", 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031993800000000003 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.844757+0000", 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013647920000000001 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.847321+0000", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025640340000000002 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.847396+0000", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4587999999999998e-05 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.847463+0000", 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.69e-05 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:678 3.13 e29/28 3:ce7f988b:::rbd_data.1124a8bc99f2.0000000000000011:head v 29'8, mlcod=29'6)", 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.843486+0000", 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.65261932199999995, 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021028069999999999, 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843486+0000", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843486+0000", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843487+0000", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.24e-06 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843497+0000", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0033e-05 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843512+0000", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4513e-05 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843523+0000", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0798000000000001e-05 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843643+0000", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119883 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843697+0000", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4586000000000002e-05 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.845518+0000", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001821217 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.845589+0000", 2024-09-22T01:14:11.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0537000000000006e-05 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:680 3.13 e29/28 3:ce2ff269:::rbd_data.1124a8bc99f2.0000000000000019:head v 29'9, mlcod=29'6)", 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:10.843939+0000", 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.65216653099999999, 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016871810000000001, 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:11.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843939+0000", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843939+0000", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843941+0000", 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-22T01:14:11.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843955+0000", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3784999999999999e-05 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843979+0000", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3974000000000001e-05 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.843995+0000", 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6577000000000001e-05 2024-09-22T01:14:11.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.844275+0000", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028005800000000002 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.844310+0000", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4316999999999997e-05 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.845593+0000", 2024-09-22T01:14:11.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012830649999999999 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:10.845626+0000", 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519000000000002e-05 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:11.594 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:11.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:14:11.596 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:11.597 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:11.597+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:11.667 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone1 intact 2024-09-22T01:14:11.711 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:11.711+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:11.772 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:11.772 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:11.773 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:11.773 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:11.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:11.821 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-22T01:14:11.822 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:11.819+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:11.899 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:11.899+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:11.953 INFO:teuthology.orchestra.run.smithi096.stdout:245 punch from 0x2944cce to 0x29494ad, (0x47df bytes) 2024-09-22T01:14:11.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:11.968 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:11.968 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:11.972 INFO:teuthology.orchestra.run.smithi096.stdout:246 trunc from 0x855fb72 to 0x11e713d 2024-09-22T01:14:11.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:12.035 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:12.035+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:12.100 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:12.099+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:12.191 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.205545540999999, 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.955312583000001, 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011381442, 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234702+0000", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234713+0000", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0507e-05 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234735+0000", 2024-09-22T01:14:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1800000000000001e-05 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234748+0000", 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3583e-05 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243350+0000", 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086011609999999995 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243483+0000", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308499999999999 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243702+0000", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021935499999999999 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245600+0000", 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018977219999999999 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245927+0000", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032715799999999997 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246017+0000", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9752000000000004e-05 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246082+0000", 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5813000000000002e-05 2024-09-22T01:14:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.955236296999999, 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013805090000000001, 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234779+0000", 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-06 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234789+0000", 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.042e-05 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234808+0000", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9256e-05 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234826+0000", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7825999999999998e-05 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244277+0000", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094506409999999992 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244411+0000", 2024-09-22T01:14:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134513 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244661+0000", 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024963399999999999 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245702+0000", 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001040736 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248290+0000", 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025886429999999998 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248450+0000", 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015967800000000001 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248582+0000", 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132316 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.683172328000001, 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011449948999999999, 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506842+0000", 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506918+0000", 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5715000000000003e-05 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506925+0000", 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4599999999999997e-06 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506929+0000", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2520000000000001e-06 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510441+0000", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035115350000000001 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510484+0000", 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510560+0000", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6184000000000001e-05 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513176+0000", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026151540000000002 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518234+0000", 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050580019999999998 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518267+0000", 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3800999999999997e-05 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518291+0000", 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3855000000000001e-05 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.683075326000001, 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011448767, 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506939+0000", 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506942+0000", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8420000000000001e-06 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506945+0000", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9349999999999996e-06 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506949+0000", 2024-09-22T01:14:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511010+0000", 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040611680000000004 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511054+0000", 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4255999999999998e-05 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511140+0000", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5841999999999996e-05 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513889+0000", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027490869999999999 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518324+0000", 2024-09-22T01:14:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044343489999999998 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518358+0000", 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4737999999999998e-05 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518387+0000", 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8572999999999999e-05 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.683002257999998, 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446914, 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507012+0000", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507015+0000", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507020+0000", 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-22T01:14:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507023+0000", 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511408+0000", 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043849700000000002 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511451+0000", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2735999999999999e-05 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511539+0000", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7766e-05 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513894+0000", 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023553100000000002 2024-09-22T01:14:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519395+0000", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055011189999999996 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519431+0000", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5457999999999998e-05 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519458+0000", 2024-09-22T01:14:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7682e-05 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.68279948, 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015085121999999999, 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507215+0000", 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507301+0000", 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6791000000000003e-05 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507310+0000", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.653e-06 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507323+0000", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2648e-05 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511896+0000", 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045732380000000003 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511954+0000", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8319000000000002e-05 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512028+0000", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513897+0000", 2024-09-22T01:14:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001869587 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522235+0000", 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083377590000000001 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522264+0000", 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9431999999999999e-05 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522299+0000", 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4860999999999997e-05 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.682635549, 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01505534, 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507379+0000", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507382+0000", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.258e-06 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507388+0000", 2024-09-22T01:14:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.263e-06 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507392+0000", 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.771e-06 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512533+0000", 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051413939999999997 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512618+0000", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4338000000000005e-05 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512742+0000", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012398199999999999 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513903+0000", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001161147 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522355+0000", 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084519360000000002 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522399+0000", 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4508000000000001e-05 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522433+0000", 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4177000000000002e-05 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.682553433999999, 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015058092, 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507461+0000", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507463+0000", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507475+0000", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.146e-05 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507479+0000", 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1049999999999997e-06 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513231+0000", 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057521810000000003 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513296+0000", 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4764999999999994e-05 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513388+0000", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1847000000000003e-05 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.516279+0000", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028914119999999999 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522465+0000", 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006186352 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522496+0000", 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0988000000000002e-05 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522518+0000", 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1792000000000001e-05 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.215332507999999, 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01429044, 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 131 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974682+0000", 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974824+0000", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014190599999999999 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974839+0000", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974849+0000", 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8369999999999992e-06 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977235+0000", 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023857219999999998 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977380+0000", 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014566199999999999 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977640+0000", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025971799999999998 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.980903+0000", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032634040000000001 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988830+0000", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260430000000007 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988904+0000", 2024-09-22T01:14:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4016000000000004e-05 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988971+0000", 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7929000000000006e-05 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.215144699, 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014424996000000001, 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974870+0000", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974876+0000", 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974889+0000", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2670000000000001e-05 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974896+0000", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9370000000000004e-06 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979031+0000", 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041348749999999997 2024-09-22T01:14:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979163+0000", 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132142 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979423+0000", 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025945100000000001 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981009+0000", 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001585801 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989179+0000", 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081698940000000005 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989243+0000", 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4480000000000006e-05 2024-09-22T01:14:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989294+0000", 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0742999999999999e-05 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.215068680000002, 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484576000000001, 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 133 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974946+0000", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0499999999999999e-06 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974952+0000", 2024-09-22T01:14:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-22T01:14:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974964+0000", 2024-09-22T01:14:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2164000000000001e-05 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974976+0000", 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1875999999999999e-05 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981087+0000", 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061111660000000003 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981224+0000", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013626900000000001 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981437+0000", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021382899999999999 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985221+0000", 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037832999999999999 2024-09-22T01:14:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989306+0000", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004085506 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989379+0000", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2909000000000003e-05 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989429+0000", 2024-09-22T01:14:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0377000000000002e-05 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214965444000001, 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018165239999999999, 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 134 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975049+0000", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975179+0000", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129722 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975197+0000", 2024-09-22T01:14:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8113000000000001e-05 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975207+0000", 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5100000000000004e-06 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982219+0000", 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070121760000000002 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982352+0000", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133249 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982617+0000", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026466800000000002 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989070+0000", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064531299999999996 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.992931+0000", 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003860774 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993092+0000", 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016092499999999999 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993213+0000", 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121836 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214789172, 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018294939, 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 135 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975225+0000", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975231+0000", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7699999999999998e-06 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975243+0000", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1644000000000001e-05 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975250+0000", 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.503e-06 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983862+0000", 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086120459999999999 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983989+0000", 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012705299999999999 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.984185+0000", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019606499999999999 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989159+0000", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049732120000000003 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993238+0000", 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040793449999999998 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993372+0000", 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013405200000000001 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993519+0000", 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733899999999999 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.276 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214747938999999, 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018936457, 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 136 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:12.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8400000000000004e-07 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975272+0000", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975284+0000", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2393e-05 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975292+0000", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.763e-06 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985342+0000", 2024-09-22T01:14:12.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010050026 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985551+0000", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208755 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985988+0000", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043766699999999998 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989513+0000", 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003525233 2024-09-22T01:14:12.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994036+0000", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045223279999999999 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994117+0000", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1549000000000001e-05 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994202+0000", 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4735999999999996e-05 2024-09-22T01:14:12.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214692103000001, 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019072927, 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975323+0000", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975483+0000", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160416 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975500+0000", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7127000000000001e-05 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975509+0000", 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4399999999999994e-06 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987086+0000", 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011576606999999999 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987414+0000", 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032788700000000001 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987806+0000", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039176100000000002 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993673+0000", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058668970000000003 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994227+0000", 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055439900000000002 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994330+0000", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010278 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994394+0000", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4542999999999995e-05 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:12.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214486620999999, 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019163698999999999, 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975528+0000", 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.017e-06 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975534+0000", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975547+0000", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2806000000000001e-05 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975554+0000", 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989583+0000", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014028175 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989829+0000", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024677100000000001 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.990235+0000", 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040547400000000001 2024-09-22T01:14:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994418+0000", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041833579999999999 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994534+0000", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011541500000000001 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994623+0000", 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9703000000000005e-05 2024-09-22T01:14:12.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994691+0000", 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7366000000000005e-05 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214446971000001, 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021194425999999999, 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975577+0000", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4669999999999993e-06 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975587+0000", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975594+0000", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1770000000000001e-06 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991392+0000", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015798764 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991632+0000", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023988199999999999 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991978+0000", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034544099999999999 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996441+0000", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004463608 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996541+0000", 2024-09-22T01:14:12.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9623000000000005e-05 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996644+0000", 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103416 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996761+0000", 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116532 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214408181, 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022802374, 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975606+0000", 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0299999999999998e-07 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975744+0000", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013788599999999999 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975767+0000", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3003e-05 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975781+0000", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.383e-05 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994785+0000", 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019003995999999999 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994930+0000", 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144683 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.995124+0000", 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019457099999999999 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998089+0000", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029647440000000001 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998221+0000", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178199999999999 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998301+0000", 2024-09-22T01:14:12.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9839999999999995e-05 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998408+0000", 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107236 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.214216779000001, 2024-09-22T01:14:12.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024814201000000001, 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:12.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975798+0000", 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975807+0000", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.533e-06 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975817+0000", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9599999999999995e-06 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975824+0000", 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3289999999999996e-06 2024-09-22T01:14:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996882+0000", 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021058038000000001 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997106+0000", 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022396800000000001 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:12.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997464+0000", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035745699999999999 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998789+0000", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001325384 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000443+0000", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001653493 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000521+0000", 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000611+0000", 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9662999999999999e-05 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.310 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:12.311 INFO:teuthology.orchestra.run.smithi096.stdout:247 write 0x8c6e279 thru 0x8c7164b (0x33d3 bytes) 2024-09-22T01:14:12.311 INFO:teuthology.orchestra.run.smithi096.stdout:248 read 0x7683410 thru 0x7689f72 (0x6b63 bytes) 2024-09-22T01:14:12.312 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:254 write 0xcb667f0 thru 0xcb71d0c (0xb51d bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:259 read 0x40a280f thru 0x40a62da (0x3acc bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:260 writesame 0x4469736 thru 0x447753a data_size 0x1(0xde05 bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:261 trunc from 0xcb71d0d to 0xaf5ff22 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:262 read 0x50ecd61 thru 0x50ed4be (0x75e bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:263 read 0x35324a7 thru 0x353580a (0x3364 bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:264 read 0x5ebda06 thru 0x5ec3804 (0x5dff bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:265 read 0x3b228be thru 0x3b2fda4 (0xd4e7 bytes) 2024-09-22T01:14:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:266 read 0x7c18f00 thru 0x7c27ba5 (0xeca6 bytes) 2024-09-22T01:14:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:269 read 0x204d124 thru 0x20520cb (0x4fa8 bytes) 2024-09-22T01:14:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:270 trunc from 0xaf5ff22 to 0x67c7b9b 2024-09-22T01:14:12.315 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:12.311+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:12.341 INFO:teuthology.orchestra.run.smithi096.stdout:271 write 0x9db3813 thru 0x9dbcafb (0x92e9 bytes) 2024-09-22T01:14:12.344 INFO:teuthology.orchestra.run.smithi096.stdout:272 write 0x25e1586 thru 0x25e1df0 (0x86b bytes) 2024-09-22T01:14:12.349 INFO:teuthology.orchestra.run.smithi096.stdout:274 trunc from 0x9dbcafc to 0xbdd79cd 2024-09-22T01:14:12.361 INFO:teuthology.orchestra.run.smithi096.stdout:276 read 0x3a30669 thru 0x3a3fc08 (0xf5a0 bytes) 2024-09-22T01:14:12.362 INFO:teuthology.orchestra.run.smithi096.stdout:277 write 0x51513c8 thru 0x516075f (0xf398 bytes) 2024-09-22T01:14:12.365 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.365 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.365 INFO:teuthology.orchestra.run.smithi096.stdout:280 write 0x7eca873 thru 0x7ecfcbc (0x544a bytes) 2024-09-22T01:14:12.366 INFO:teuthology.orchestra.run.smithi096.stdout:281 read 0x640628f thru 0x6411e53 (0xbbc5 bytes) 2024-09-22T01:14:12.367 INFO:teuthology.orchestra.run.smithi096.stdout:282 write 0x68029c4 thru 0x680e882 (0xbebf bytes) 2024-09-22T01:14:12.370 INFO:teuthology.orchestra.run.smithi096.stdout:283 write 0x5e4f2a2 thru 0x5e59600 (0xa35f bytes) 2024-09-22T01:14:12.372 INFO:teuthology.orchestra.run.smithi096.stdout:284 read 0xbf9b2 thru 0xc8a8a (0x90d9 bytes) 2024-09-22T01:14:12.373 INFO:teuthology.orchestra.run.smithi096.stdout:285 writesame 0xa4cd253 thru 0xa4db4f6 data_size 0x1e(0xe2a4 bytes) 2024-09-22T01:14:12.377 INFO:teuthology.orchestra.run.smithi096.stdout:287 read 0x29e9313 thru 0x29f22ef (0x8fdd bytes) 2024-09-22T01:14:12.377 INFO:teuthology.orchestra.run.smithi096.stdout:288 write 0x50ce9a6 thru 0x50d6481 (0x7adc bytes) 2024-09-22T01:14:12.379 INFO:teuthology.orchestra.run.smithi096.stdout:289 write 0xc0b1bf thru 0xc0deb0 (0x2cf2 bytes) 2024-09-22T01:14:12.381 INFO:teuthology.orchestra.run.smithi096.stdout:290 writesame 0xa2be70c thru 0xa2be8ab data_size 0xd(0x1a0 bytes) 2024-09-22T01:14:12.385 INFO:teuthology.orchestra.run.smithi096.stdout:292 punch from 0x4dc212 to 0x4e8aaf, (0xc89d bytes) 2024-09-22T01:14:12.386 INFO:teuthology.orchestra.run.smithi096.stdout:293 write 0x88edc76 thru 0x88f0964 (0x2cef bytes) 2024-09-22T01:14:12.387 INFO:teuthology.orchestra.run.smithi096.stdout:294 writesame 0xf920ae thru 0xf966e4 data_size 0x5(0x4637 bytes) 2024-09-22T01:14:12.394 INFO:teuthology.orchestra.run.smithi096.stdout:295 writesame 0x14960de thru 0x14a3758 data_size 0x1(0xd67b bytes) 2024-09-22T01:14:12.412 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:12.413+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:12.462 INFO:teuthology.orchestra.run.smithi096.stdout:296 write 0xcebe50f thru 0xcebe980 (0x472 bytes) 2024-09-22T01:14:12.466 INFO:teuthology.orchestra.run.smithi096.stdout:297 punch from 0x8991639 to 0x89939ad, (0x2374 bytes) 2024-09-22T01:14:12.466 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.466 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.466 INFO:teuthology.orchestra.run.smithi096.stdout:300 write 0x9f3b4ce thru 0x9f40277 (0x4daa bytes) 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:4073 3.11 e30/28 3:8d4f5db2:::rbd_data.1124ef2748ce.0000000000000009:head v 30'12, mlcod=30'10)", 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.469049+0000", 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.003698986, 2024-09-22T01:14:12.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003714803, 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469049+0000", 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469049+0000", 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469050+0000", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.128e-06 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469059+0000", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8030000000000003e-06 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469076+0000", 2024-09-22T01:14:12.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7034e-05 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469086+0000", 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5599999999999999e-06 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469185+0000", 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9154000000000006e-05 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.469226+0000", 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1071e-05 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-22T01:14:12.475 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:12.486 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:12.487 INFO:teuthology.orchestra.run.smithi096.stdout:301 writesame 0xd7cfde1 thru 0xd7d9414 data_size 0x2(0x9634 bytes) 2024-09-22T01:14:12.488 INFO:teuthology.orchestra.run.smithi096.stdout:303 read 0x246e9bf thru 0x2472002 (0x3644 bytes) 2024-09-22T01:14:12.489 INFO:teuthology.orchestra.run.smithi096.stdout:304 read 0xa3239d9 thru 0xa332b85 (0xf1ad bytes) 2024-09-22T01:14:12.490 INFO:teuthology.orchestra.run.smithi096.stdout:305 read 0xd45c87b thru 0xd460bd2 (0x4358 bytes) 2024-09-22T01:14:12.490 INFO:teuthology.orchestra.run.smithi096.stdout:306 read 0xceac617 thru 0xceb07e3 (0x41cd bytes) 2024-09-22T01:14:12.491 INFO:teuthology.orchestra.run.smithi096.stdout:307 writesame 0x4cd5e0e thru 0x4cdbe45 data_size 0x4(0x6038 bytes) 2024-09-22T01:14:12.498 INFO:teuthology.orchestra.run.smithi096.stdout:308 writesame 0x8c17c4e thru 0x8c1b9a1 data_size 0x32(0x3d54 bytes) 2024-09-22T01:14:12.501 INFO:teuthology.orchestra.run.smithi096.stdout:309 punch from 0xac21dbc to 0xac287c1, (0x6a05 bytes) 2024-09-22T01:14:12.501 INFO:teuthology.orchestra.run.smithi096.stdout:310 punch from 0x36721fa to 0x3680280, (0xe086 bytes) 2024-09-22T01:14:12.501 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.501 INFO:teuthology.orchestra.run.smithi096.stdout:312 write 0xc41ca10 thru 0xc42c94b (0xff3c bytes) 2024-09-22T01:14:12.513 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:12.513+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:12.519 INFO:teuthology.orchestra.run.smithi096.stdout:313 trunc from 0xd7d9415 to 0x4113b8e 2024-09-22T01:14:12.542 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xd4f7d55 2024-09-22T01:14:12.555 INFO:teuthology.orchestra.run.smithi096.stdout:315 trunc from 0x4113b8e to 0xd4f7d55 2024-09-22T01:14:12.566 INFO:teuthology.orchestra.run.smithi096.stdout:316 read 0x4f7e339 thru 0x4f8db97 (0xf85f bytes) 2024-09-22T01:14:12.581 INFO:teuthology.orchestra.run.smithi096.stdout:318 write 0xd763924 thru 0xd769f0c (0x65e9 bytes) 2024-09-22T01:14:12.583 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:14:12.590 INFO:teuthology.orchestra.run.smithi096.stdout:319 write 0x257bb32 thru 0x25846aa (0x8b79 bytes) 2024-09-22T01:14:12.599 INFO:teuthology.orchestra.run.smithi096.stdout:321 writesame 0x3e70610 thru 0x3e779ff data_size 0x70(0x73f0 bytes) 2024-09-22T01:14:12.601 INFO:teuthology.orchestra.run.smithi096.stdout:322 write 0x6688cba thru 0x6691812 (0x8b59 bytes) 2024-09-22T01:14:12.607 INFO:teuthology.orchestra.run.smithi096.stdout:323 trunc from 0xd769f0d to 0xb739e4a 2024-09-22T01:14:12.611 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:12.611 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:12.612 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:12.612 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:12.612 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:12.614 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:12.611+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:12.623 INFO:teuthology.orchestra.run.smithi096.stdout:324 write 0x191e1bb thru 0x192c12c (0xdf72 bytes) 2024-09-22T01:14:12.625 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:12.628 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.628 INFO:teuthology.orchestra.run.smithi096.stdout:328 punch from 0x1beef56 to 0x1bfc78e, (0xd838 bytes) 2024-09-22T01:14:12.628 INFO:teuthology.orchestra.run.smithi096.stdout:329 read 0x2da2237 thru 0x2dab33d (0x9107 bytes) 2024-09-22T01:14:12.628 INFO:teuthology.orchestra.run.smithi096.stdout:331 write 0xb71fae thru 0xb7e149 (0xc19c bytes) 2024-09-22T01:14:12.651 INFO:teuthology.orchestra.run.smithi096.stdout:332 write 0xe933993 thru 0xe939753 (0x5dc1 bytes) 2024-09-22T01:14:12.652 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xea9ddc2 2024-09-22T01:14:12.652 INFO:teuthology.orchestra.run.smithi096.stdout:333 trunc from 0xe939754 to 0xea9ddc2 2024-09-22T01:14:12.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:12.663 INFO:teuthology.orchestra.run.smithi096.stdout:334 write 0xbbdec29 thru 0xbbe9fce (0xb3a6 bytes) 2024-09-22T01:14:12.668 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.668 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.668 INFO:teuthology.orchestra.run.smithi096.stdout:337 writesame 0x4d390bb thru 0x4d3c3ce data_size 0xe(0x3314 bytes) 2024-09-22T01:14:12.671 INFO:teuthology.orchestra.run.smithi096.stdout:338 read 0x177511f thru 0x177a0d3 (0x4fb5 bytes) 2024-09-22T01:14:12.672 INFO:teuthology.orchestra.run.smithi096.stdout:339 writesame 0x5df924e thru 0x5dfafae data_size 0x17(0x1d61 bytes) 2024-09-22T01:14:12.676 INFO:teuthology.orchestra.run.smithi096.stdout:340 read 0xbf51996 thru 0xbf59770 (0x7ddb bytes) 2024-09-22T01:14:12.677 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.677 INFO:teuthology.orchestra.run.smithi096.stdout:342 write 0xa982b18 thru 0xa98bdff (0x92e8 bytes) 2024-09-22T01:14:12.681 INFO:teuthology.orchestra.run.smithi096.stdout:343 write 0xcd78c5a thru 0xcd8321f (0xa5c6 bytes) 2024-09-22T01:14:12.686 INFO:teuthology.orchestra.run.smithi096.stdout:345 read 0xc16c025 thru 0xc16d414 (0x13f0 bytes) 2024-09-22T01:14:12.687 INFO:teuthology.orchestra.run.smithi096.stdout:346 read 0x6ad1ac4 thru 0x6adec53 (0xd190 bytes) 2024-09-22T01:14:12.688 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.688 INFO:teuthology.orchestra.run.smithi096.stdout:349 read 0xabad6b5 thru 0xabafb11 (0x245d bytes) 2024-09-22T01:14:12.688 INFO:teuthology.orchestra.run.smithi096.stdout:350 write 0x1cf5e45 thru 0x1d04c3b (0xedf7 bytes) 2024-09-22T01:14:12.692 INFO:teuthology.orchestra.run.smithi096.stdout:352 read 0x8d5b1ba thru 0x8d6b16d (0xffb4 bytes) 2024-09-22T01:14:12.693 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.709 INFO:teuthology.orchestra.run.smithi096.stdout:355 trunc from 0xea9ddc2 to 0x353b63c 2024-09-22T01:14:12.715 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:12.713+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:12.727 INFO:teuthology.orchestra.run.smithi096.stdout:356 read 0x2a9f559 thru 0x2aaa243 (0xaceb bytes) 2024-09-22T01:14:12.752 INFO:teuthology.orchestra.run.smithi096.stdout:357 write 0xca834e6 thru 0xca83ed8 (0x9f3 bytes) 2024-09-22T01:14:12.755 INFO:teuthology.orchestra.run.smithi096.stdout:358 read 0x4821df8 thru 0x482768d (0x5896 bytes) 2024-09-22T01:14:12.761 INFO:teuthology.orchestra.run.smithi096.stdout:361 trunc from 0xca83ed9 to 0x93650c1 2024-09-22T01:14:12.767 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1738 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 131072~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76136134300000002, 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014237117000000001, 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1738 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006357+0000", 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5000999999999999e-05 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006366+0000", 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006372+0000", 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8379999999999998e-06 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013432+0000", 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070601079999999998 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013499+0000", 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7531000000000001e-05 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020579+0000", 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070795700000000003 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2025 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2916352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76067048699999995, 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013872165000000001, 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2025 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007035+0000", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9309999999999998e-06 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007042+0000", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8369999999999999e-06 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007046+0000", 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020772+0000", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013725342 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020840+0000", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8576000000000002e-05 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020905+0000", 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4469999999999998e-05 2024-09-22T01:14:12.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2026 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76065001899999996, 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014009057, 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2026 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007056+0000", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007062+0000", 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5570000000000001e-06 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007067+0000", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8029999999999996e-06 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020933+0000", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013865475 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020999+0000", 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390999999999995e-05 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021062+0000", 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2027 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76063022400000002, 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014158201, 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2027 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007074+0000", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007083+0000", 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9889999999999998e-06 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007089+0000", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5259999999999999e-06 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007094+0000", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6530000000000002e-06 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021090+0000", 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01399624 2024-09-22T01:14:12.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021157+0000", 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6538999999999994e-05 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021231+0000", 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4640000000000004e-05 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2028 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3801088~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76060270799999996, 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014289873, 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2028 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007101+0000", 2024-09-22T01:14:12.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007103+0000", 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007111+0000", 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007122+0000", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988000000000001e-05 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021258+0000", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014136319 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021326+0000", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7642000000000001e-05 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021390+0000", 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4645000000000002e-05 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76057463400000003, 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:12.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:12.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:12.794 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76055396500000005, 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:12.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:12.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:12.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76052365200000005, 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:12.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:12.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:12.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76050439800000003, 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76048629599999995, 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:12.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76046557299999995, 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:12.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:12.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76044508399999999, 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.816 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:12.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:12.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76042364200000001, 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:12.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:12.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76039279500000001, 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76035526899999994, 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:12.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76032314099999998, 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.837 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.76030013399999996, 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.72098630100000005, 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.719699121, 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:12.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:12.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.71935493800000005, 2024-09-22T01:14:12.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:12.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:12.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:12.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:12.858 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:12.859 INFO:teuthology.orchestra.run.smithi096.stdout:362 punch from 0x8c6ca2c to 0x8c79bb3, (0xd187 bytes) 2024-09-22T01:14:12.859 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.859 INFO:teuthology.orchestra.run.smithi096.stdout:365 punch from 0x28eca4f to 0x28f5f32, (0x94e3 bytes) 2024-09-22T01:14:12.859 INFO:teuthology.orchestra.run.smithi096.stdout:368 read 0x1948357 thru 0x194b7a1 (0x344b bytes) 2024-09-22T01:14:12.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:12.861 INFO:teuthology.orchestra.run.smithi096.stdout:369 write 0xd69eef8 thru 0xd6a6aa9 (0x7bb2 bytes) 2024-09-22T01:14:12.861 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:12.861 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:12.862 INFO:teuthology.orchestra.run.smithi096.stdout:370 trunc from 0xd6a6aaa to 0x95cd4e2 2024-09-22T01:14:12.862 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.862 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:12.862 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:12.862 INFO:teuthology.orchestra.run.smithi096.stdout:373 write 0xb56d9c9 thru 0xb576eb9 (0x94f1 bytes) 2024-09-22T01:14:12.863 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:12.863 INFO:teuthology.orchestra.run.smithi096.stdout:375 writesame 0xc21dee6 thru 0xc223b05 data_size 0x58(0x5c20 bytes) 2024-09-22T01:14:12.864 INFO:teuthology.orchestra.run.smithi096.stdout:376 trunc from 0xc223b06 to 0x94e9101 2024-09-22T01:14:12.864 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:12.859+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:12.864 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:12.864 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:12.865 INFO:teuthology.orchestra.run.smithi096.stdout:377 trunc from 0x94e9101 to 0x950b111 2024-09-22T01:14:12.874 INFO:teuthology.orchestra.run.smithi096.stdout:378 read 0x491cf74 thru 0x4926d15 (0x9da2 bytes) 2024-09-22T01:14:12.874 INFO:teuthology.orchestra.run.smithi096.stdout:379 read 0x48952ee thru 0x489be8a (0x6b9d bytes) 2024-09-22T01:14:12.881 INFO:teuthology.orchestra.run.smithi096.stdout:380 trunc from 0x950b111 to 0xe25d74e 2024-09-22T01:14:12.884 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:12.884 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:12.901 INFO:teuthology.orchestra.run.smithi096.stdout:381 write 0xebdbdb3 thru 0xebe9f94 (0xe1e2 bytes) 2024-09-22T01:14:12.903 INFO:teuthology.orchestra.run.smithi096.stdout:382 punch from 0xc68e311 to 0xc69280f, (0x44fe bytes) 2024-09-22T01:14:12.903 INFO:teuthology.orchestra.run.smithi096.stdout:383 read 0xb285083 thru 0xb28c8ba (0x7838 bytes) 2024-09-22T01:14:12.904 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:12.904 INFO:teuthology.orchestra.run.smithi096.stdout:385 write 0x53771e0 thru 0x538398e (0xc7af bytes) 2024-09-22T01:14:12.908 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:12.908 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:12.921 INFO:teuthology.orchestra.run.smithi096.stdout:386 trunc from 0xebe9f95 to 0x285a0e3 2024-09-22T01:14:12.932 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:12.932 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:12.938 INFO:teuthology.orchestra.run.smithi096.stdout:387 read 0x23bbace thru 0x23c0830 (0x4d63 bytes) 2024-09-22T01:14:12.955 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:12.955 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:12.957 INFO:teuthology.orchestra.run.smithi096.stdout:388 write 0x79753a9 thru 0x7979c93 (0x48eb bytes) 2024-09-22T01:14:12.960 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:12.961+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:12.973 INFO:teuthology.orchestra.run.smithi096.stdout:392 write 0x881c3e8 thru 0x88288a0 (0xc4b9 bytes) 2024-09-22T01:14:12.988 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:12.988 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:12.995 INFO:teuthology.orchestra.run.smithi096.stdout:394 write 0xe715eb0 thru 0xe7209bd (0xab0e bytes) 2024-09-22T01:14:13.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:13.006 INFO:teuthology.orchestra.run.smithi096.stdout:396 trunc from 0xe7209be to 0x97a634b 2024-09-22T01:14:13.040 INFO:teuthology.orchestra.run.smithi096.stdout:397 writesame 0xcbc00af thru 0xcbcec49 data_size 0x5(0xeb9b bytes) 2024-09-22T01:14:13.041 INFO:teuthology.orchestra.run.smithi096.stdout:398 read 0x859fcc7 thru 0x85a5c8c (0x5fc6 bytes) 2024-09-22T01:14:13.042 INFO:teuthology.orchestra.run.smithi096.stdout:399 read 0x207942e thru 0x207b928 (0x24fb bytes) 2024-09-22T01:14:13.043 INFO:teuthology.orchestra.run.smithi096.stdout:400 writesame 0x2d1daf7 thru 0x2d25fa0 data_size 0x1(0x84aa bytes) 2024-09-22T01:14:13.061 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:13.059+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:13.083 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:13.083 INFO:teuthology.orchestra.run.smithi096.stdout:404 write 0x6c82617 thru 0x6c850eb (0x2ad5 bytes) 2024-09-22T01:14:13.103 INFO:teuthology.orchestra.run.smithi096.stdout:405 write 0xcda2c65 thru 0xcdaa521 (0x78bd bytes) 2024-09-22T01:14:13.121 INFO:teuthology.orchestra.run.smithi096.stdout:407 writesame 0xd9e1341 thru 0xd9eca8e data_size 0x9e(0xb74e bytes) 2024-09-22T01:14:13.121 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.121 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.123 INFO:teuthology.orchestra.run.smithi096.stdout:408 read 0x3864454 thru 0x38650bd (0xc6a bytes) 2024-09-22T01:14:13.126 INFO:teuthology.orchestra.run.smithi096.stdout:410 trunc from 0xd9eca8f to 0xc81f755 2024-09-22T01:14:13.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:13.151 INFO:teuthology.orchestra.run.smithi096.stdout:411 clone 3 order 22 su 1048576 sc 6 2024-09-22T01:14:13.163 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:13.161+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:13.263 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:13.265+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:13.270 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.270 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.284 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:13.364 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:13.363+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:13.414 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.414 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:13.465 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:13.463+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:13.551 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.551 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.566 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:13.563+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:13.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:13.666 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:13.665+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:13.698 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.698 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.713 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:13.743 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone2 from 0xc81f755 (overlap 0x11e713d) to 0xc88277 2024-09-22T01:14:13.767 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:13.767+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:13.868 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:13.867+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:13.883 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:13.883 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:13.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:13.902 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-22T01:14:13.969 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:13.969+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:14.020 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:14.020 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:14.035 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:14.069 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:14.067+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:14.171 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:14.169+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:14.200 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:14.201 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:14.215 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:14.256 INFO:teuthology.orchestra.run.smithi096.stdout:412 punch from 0x2e57cfb to 0x2e6763e, (0xf943 bytes) 2024-09-22T01:14:14.262 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.262 INFO:teuthology.orchestra.run.smithi096.stdout:416 write 0x64d397e thru 0x64e1b70 (0xe1f3 bytes) 2024-09-22T01:14:14.272 INFO:teuthology.orchestra.run.smithi096.stdout:417 write 0xa43ad23 thru 0xa43cd64 (0x2042 bytes) 2024-09-22T01:14:14.284 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.284 INFO:teuthology.orchestra.run.smithi096.stdout:420 read 0xa64023a thru 0xa646338 (0x60ff bytes) 2024-09-22T01:14:14.284 INFO:teuthology.orchestra.run.smithi096.stdout:421 read 0x8098d4c thru 0x809eda3 (0x6058 bytes) 2024-09-22T01:14:14.293 INFO:teuthology.orchestra.run.smithi096.stdout:422 writesame 0xc856471 thru 0xc85f397 data_size 0x1(0x8f27 bytes) 2024-09-22T01:14:14.299 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:14.301+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:14.300 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.300 INFO:teuthology.orchestra.run.smithi096.stdout:424 punch from 0xbc7e63f to 0xbc8372b, (0x50ec bytes) 2024-09-22T01:14:14.300 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.303 INFO:teuthology.orchestra.run.smithi096.stdout:428 trunc from 0xc85f398 to 0xe884cf5 2024-09-22T01:14:14.312 INFO:teuthology.orchestra.run.smithi096.stdout:429 read 0x8b31d3e thru 0x8b3a452 (0x8715 bytes) 2024-09-22T01:14:14.336 INFO:teuthology.orchestra.run.smithi096.stdout:431 trunc from 0xe884cf5 to 0x1cc595b 2024-09-22T01:14:14.372 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:14.373+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:14.382 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:14.382 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:14.384 INFO:teuthology.orchestra.run.smithi096.stdout:432 read 0x970dc5 thru 0x97f949 (0xeb85 bytes) 2024-09-22T01:14:14.385 INFO:teuthology.orchestra.run.smithi096.stdout:433 read 0x485c14 thru 0x4871ce (0x15bb bytes) 2024-09-22T01:14:14.386 INFO:teuthology.orchestra.run.smithi096.stdout:434 read 0x129c53d thru 0x12a8f1b (0xc9df bytes) 2024-09-22T01:14:14.386 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.386 INFO:teuthology.orchestra.run.smithi096.stdout:436 punch from 0xad216 to 0xaecaf, (0x1a99 bytes) 2024-09-22T01:14:14.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:14.399 INFO:teuthology.orchestra.run.smithi096.stdout:437 trunc from 0x1cc595b to 0x8bb30e8 2024-09-22T01:14:14.410 INFO:teuthology.orchestra.run.smithi096.stdout:439 read 0x21319ad thru 0x213f58e (0xdbe2 bytes) 2024-09-22T01:14:14.411 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.411 INFO:teuthology.orchestra.run.smithi096.stdout:443 write 0x73a023d thru 0x73a6414 (0x61d8 bytes) 2024-09-22T01:14:14.417 INFO:teuthology.orchestra.run.smithi096.stdout:444 punch from 0x5374b36 to 0x537acb8, (0x6182 bytes) 2024-09-22T01:14:14.417 INFO:teuthology.orchestra.run.smithi096.stdout:446 clone 4 order 21 su 65536 sc 15 2024-09-22T01:14:14.484 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:14.485+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:14.555 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:14.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:14.555 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:14:14.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:14:14.570 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:14.570 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:14.573 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:14.573+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:14.585 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:14.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:14:14.634 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone3 from 0x8bb30e8 (overlap 0x1cc595b) to 0x114359d 2024-09-22T01:14:14.675 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:14.677+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:14.757 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:14.757 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:14.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:14.797 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-22T01:14:14.816 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:14.817+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:14.829 INFO:teuthology.orchestra.run.smithi096.stdout:448 read 0x7889bf5 thru 0x7889ee6 (0x2f2 bytes) 2024-09-22T01:14:14.830 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.830 INFO:teuthology.orchestra.run.smithi096.stdout:450 flatten 2024-09-22T01:14:14.875 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:14.877+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:14.913 INFO:teuthology.orchestra.run.smithi096.stdout:453 punch from 0x4357917 to 0x4365121, (0xd80a bytes) 2024-09-22T01:14:14.913 INFO:teuthology.orchestra.run.smithi096.stdout:454 read 0x6a1dfa7 thru 0x6a27da0 (0x9dfa bytes) 2024-09-22T01:14:14.914 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:14.927 INFO:teuthology.orchestra.run.smithi096.stdout:456 trunc from 0x8bb30e8 to 0x694f9c 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 33, 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:14:14.491394+0000", 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:14:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:09.513080+0000", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5799936, 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:14:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "jewel", 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:14:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:14:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:14:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:14:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:14:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:14:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:14:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:14:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:14:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:14:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:14:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:14:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 4.9800000190734863, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 4.9800000190734863, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.87999999523162842, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 4.380000114440918, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 4.380000114440918, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:14:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:14:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:14:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "33", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 6, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 33, 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:14:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:14:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:14:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:14:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:14:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:14:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:14:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 28, 2024-09-22T01:14:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:14:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:14:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:14:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 28, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:14:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:14:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:14:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:14:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 28, 2024-09-22T01:14:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:14:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:14:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:14:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 28, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:14:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:14:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:14:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:14:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:14:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:14:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [], 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:14:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [], 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [], 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [ 2024-09-22T01:14:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 2, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:14:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:14:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:14:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:15.013 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:15.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:15.014 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:15.014 INFO:teuthology.orchestra.run.smithi096.stdout:truncating to largest ever: 0xee6299e 2024-09-22T01:14:15.014 INFO:teuthology.orchestra.run.smithi096.stdout:458 trunc from 0x694f9c to 0xee6299e 2024-09-22T01:14:15.015 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:15.015 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:15.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:15.017 INFO:tasks.thrashosds.thrasher:j is {'epoch': 33, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:14:14.491394+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:09.513080+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 28, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:14:15.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:459 read 0x269b9e8 thru 0x269f1ad (0x37c6 bytes) 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:460 trunc from 0xee6299e to 0x8aa1526 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:463 trunc from 0x8aa1526 to 0x4050122 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:465 read 0x114a757 thru 0x114d4e1 (0x2d8b bytes) 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:466 punch from 0x2a102d8 to 0x2a152a3, (0x4fcb bytes) 2024-09-22T01:14:15.018 INFO:teuthology.orchestra.run.smithi096.stdout:467 trunc from 0x4050122 to 0x8cd484a 2024-09-22T01:14:15.018 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:15.011+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:15.021 INFO:teuthology.orchestra.run.smithi096.stdout:468 punch from 0x7ac448a to 0x7acfb23, (0xb699 bytes) 2024-09-22T01:14:15.022 INFO:teuthology.orchestra.run.smithi096.stdout:469 write 0x5596ee1 thru 0x559958f (0x26af bytes) 2024-09-22T01:14:15.026 INFO:teuthology.orchestra.run.smithi096.stdout:470 clone 5 order 21 su 32768 sc 12 2024-09-22T01:14:15.115 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:15.115+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:15.158 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:15.158 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:15.175 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:15.175 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:15.186 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:15.186 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:15.192 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:15.192 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:15.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:15.208 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:15.208 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:15.216 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:15.215+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:15.228 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:15.228 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:15.245 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:15.264 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:15.264 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:15.282 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:15.283 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:15.316 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:15.317 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:15.317 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:15.315+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:15.323 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:676 3.0 e29/24 3:0088113c:::rbd_data.1124a8bc99f2.000000000000000f:head v 29'9, mlcod=29'7)", 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.4803275449999997, 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0032775980000000001, 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844360+0000", 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844362+0000", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7770000000000001e-06 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844375+0000", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844406+0000", 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269000000000002e-05 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844419+0000", 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.331e-05 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844501+0000", 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1818999999999997e-05 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844529+0000", 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8408e-05 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.847523+0000", 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029937520000000001 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.847638+0000", 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011443299999999999 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:679 3.c e29/24 3:33070f00:::rbd_data.1124a8bc99f2.0000000000000012:head v 29'5, mlcod=29'3)", 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.4798263770000002, 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033864889999999999, 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844861+0000", 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844862+0000", 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2099999999999995e-07 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844870+0000", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8860000000000004e-06 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844882+0000", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1917e-05 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844890+0000", 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5210000000000002e-06 2024-09-22T01:14:15.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844955+0000", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4814999999999995e-05 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.844979+0000", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3442e-05 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.848169+0000", 2024-09-22T01:14:15.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003190657 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:10.848248+0000", 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8330000000000004e-05 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:695 3.0 e30/24 3:05f5f002:::rbd_object_map.1124a8bc99f2.0000000000000004:head v 30'10, mlcod=29'8)", 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:11.553672+0000", 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.7710159829999998, 2024-09-22T01:14:15.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024081979999999998, 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553672+0000", 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553672+0000", 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553672+0000", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8999999999999996e-07 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553681+0000", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553724+0000", 2024-09-22T01:14:15.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2876000000000001e-05 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553746+0000", 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2283e-05 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553913+0000", 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000166482 2024-09-22T01:14:15.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.553978+0000", 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5179999999999996e-05 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.555996+0000", 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020175670000000001 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:11.556080+0000", 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4099999999999998e-05 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 30/24 rep_tid 79 entries 30'4 (0'0) error 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head by client.4388.0:3979 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'1)", 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.273124+0000", 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.0515638190000001, 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0026324859999999999, 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273124+0000", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273124+0000", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273126+0000", 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7829999999999999e-06 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273147+0000", 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1257e-05 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273203+0000", 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5912999999999998e-05 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273248+0000", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5383000000000002e-05 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.273408+0000", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016025600000000001 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.275756+0000", 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023478940000000001 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 30/24 rep_tid 81 entries 30'5 (0'0) error 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head by client.4388.0:4004 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'3)", 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.304161+0000", 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.0205267390000001, 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033730840000000002, 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304161+0000", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304161+0000", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304163+0000", 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304185+0000", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1886999999999999e-05 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304262+0000", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7279000000000003e-05 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304316+0000", 2024-09-22T01:14:15.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4483e-05 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.304517+0000", 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020060499999999999 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.307534+0000", 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0030168230000000001 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 30/24 rep_tid 83 entries 30'6 (0'0) error 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head by client.4388.0:4007 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'3)", 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.305083+0000", 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.0196043690000001, 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031440119999999998, 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305083+0000", 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305083+0000", 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305085+0000", 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5430000000000001e-06 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305107+0000", 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2234000000000001e-05 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305165+0000", 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7855000000000001e-05 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305208+0000", 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3563999999999998e-05 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.305421+0000", 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021218800000000001 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.308227+0000", 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0028066279999999998 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4059 3.8 e30/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 30'8, mlcod=30'6)", 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.393860+0000", 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.9308273009999999, 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021219099999999999, 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393860+0000", 2024-09-22T01:14:15.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393860+0000", 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393862+0000", 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5430000000000001e-06 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393931+0000", 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9633000000000002e-05 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393960+0000", 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8353000000000001e-05 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.393973+0000", 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3356999999999999e-05 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.394103+0000", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013009499999999999 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.394161+0000", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8035999999999998e-05 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.395906+0000", 2024-09-22T01:14:15.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001744654 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.395982+0000", 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6239e-05 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4096 3.8 e30/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 30'12, mlcod=30'9)", 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.528694+0000", 2024-09-22T01:14:15.354 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.7959940429999999, 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002821156, 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528694+0000", 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528694+0000", 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528695+0000", 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3999999999999999e-06 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528707+0000", 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2480000000000001e-05 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528734+0000", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6185000000000001e-05 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528751+0000", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7186000000000001e-05 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528872+0000", 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012129300000000001 2024-09-22T01:14:15.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.528923+0000", 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1044999999999997e-05 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.531424+0000", 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025010710000000001 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.531515+0000", 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0495999999999995e-05 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4135 3.8 e31/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 31'13, mlcod=30'11)", 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.597940+0000", 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.7267472719999999, 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037508659999999998, 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.597940+0000", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.597940+0000", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.597941+0000", 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4000000000000001e-07 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.598005+0000", 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3971000000000001e-05 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.598030+0000", 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5259999999999999e-05 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.598048+0000", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8179999999999999e-05 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.598220+0000", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017136899999999999 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.598287+0000", 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7013000000000004e-05 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.601587+0000", 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033002510000000001 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.601691+0000", 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000104082 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 31/24 rep_tid 101 entries 31'14 (0'0) error 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head by client.4388.0:4138 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'12)", 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.611755+0000", 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.7129322089999999, 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020743060000000002, 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611755+0000", 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611755+0000", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611757+0000", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7859999999999999e-06 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611780+0000", 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2370000000000001e-05 2024-09-22T01:14:15.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611839+0000", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9129000000000003e-05 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.611879+0000", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0670000000000002e-05 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.612076+0000", 2024-09-22T01:14:15.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019627300000000001 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.613830+0000", 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017540780000000001 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 31/24 rep_tid 103 entries 31'15 (0'0) error 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head by client.4388.0:4141 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'12)", 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.613988+0000", 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.7107000389999998, 2024-09-22T01:14:15.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023170510000000001, 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.613988+0000", 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.613988+0000", 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.613989+0000", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.212e-06 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.614005+0000", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6036999999999999e-05 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.614044+0000", 2024-09-22T01:14:15.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.614079+0000", 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4925999999999997e-05 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.614318+0000", 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023929499999999999 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.616305+0000", 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019865389999999998 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4161 3.8 e31/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 31'17, mlcod=31'15)", 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.632367+0000", 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.6923203949999999, 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023491150000000001, 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632367+0000", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632367+0000", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632369+0000", 2024-09-22T01:14:15.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.547e-06 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632434+0000", 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5388999999999993e-05 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632461+0000", 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6593000000000001e-05 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632474+0000", 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2836999999999999e-05 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632604+0000", 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013094499999999999 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.632668+0000", 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3769999999999994e-05 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.634604+0000", 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001935932 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.634716+0000", 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011210200000000001 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4185 3.8 e31/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 31'18, mlcod=31'16)", 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.676903+0000", 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.6477846989999998, 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020401410000000001, 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.676903+0000", 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.676903+0000", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.676905+0000", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.733e-06 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.676970+0000", 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5170000000000001e-05 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.677001+0000", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0777e-05 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.677019+0000", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8889000000000002e-05 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.677129+0000", 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010997000000000001 2024-09-22T01:14:15.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.677169+0000", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9308999999999999e-05 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.678838+0000", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016693599999999999 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.678943+0000", 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010493300000000001 2024-09-22T01:14:15.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4187 3.8 e31/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 31'19, mlcod=31'17)", 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.681808+0000", 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.642879277, 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002255473, 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681808+0000", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681808+0000", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681810+0000", 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6169999999999999e-06 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681871+0000", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1116000000000002e-05 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681901+0000", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0049999999999999e-05 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.681923+0000", 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1588999999999999e-05 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.682060+0000", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013706600000000001 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.682121+0000", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0810000000000002e-05 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.683981+0000", 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001860235 2024-09-22T01:14:15.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.684064+0000", 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2990000000000003e-05 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4193 3.8 e31/24 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head v 31'20, mlcod=31'18)", 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.692879+0000", 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.6318083720000001, 2024-09-22T01:14:15.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020035510000000001, 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692879+0000", 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692879+0000", 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692881+0000", 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5999999999999999e-06 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692942+0000", 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1488999999999994e-05 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692972+0000", 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.01e-05 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.692997+0000", 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.495e-05 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.693139+0000", 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014155600000000001 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.693179+0000", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0246000000000001e-05 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.694805+0000", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016256269999999999 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.694883+0000", 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7983000000000002e-05 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.388 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 31/24 rep_tid 114 entries 31'21 (0'0) error 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head by client.4388.0:4197 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'19)", 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.713258+0000", 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.6114293100000001, 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023316919999999998, 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713258+0000", 2024-09-22T01:14:15.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713258+0000", 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713260+0000", 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.793e-06 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713281+0000", 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1183999999999999e-05 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713390+0000", 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108545 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713431+0000", 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1510000000000001e-05 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.713628+0000", 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019633199999999999 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.715590+0000", 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0019623280000000002 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "pg_update_log_missing(3.8 epoch 31/24 rep_tid 120 entries 31'24 (0'0) error 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head by client.4388.0:4230 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 31'22)", 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.765485+0000", 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.5592021259999997, 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0020099509999999998, 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765485+0000", 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765485+0000", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765487+0000", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765509+0000", 2024-09-22T01:14:15.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.156e-05 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765602+0000", 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3076000000000001e-05 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765644+0000", 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2036000000000002e-05 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.765887+0000", 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000242618 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.767495+0000", 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0016087479999999999 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4262 3.8 e31/24 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head v 31'27, mlcod=31'25)", 2024-09-22T01:14:15.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.809523+0000", 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.515164253, 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023181690000000001, 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809523+0000", 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809523+0000", 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809525+0000", 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0870000000000001e-06 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809541+0000", 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5156e-05 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809578+0000", 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7636999999999999e-05 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809602+0000", 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3450000000000001e-05 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809779+0000", 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017764500000000001 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.809836+0000", 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6450000000000003e-05 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.811726+0000", 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001889845 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.811841+0000", 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115899 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4358 3.8 e31/24 3:163e17f8:::rbd_data.1124ef2748ce.0000000000000000:head v 31'31, mlcod=31'29)", 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.925658+0000", 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.3990296249999998, 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022608120000000001, 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925658+0000", 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925658+0000", 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925660+0000", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.063e-06 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925676+0000", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5795999999999999e-05 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925714+0000", 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8287000000000001e-05 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925736+0000", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2079999999999999e-05 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925892+0000", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015559499999999999 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.925953+0000", 2024-09-22T01:14:15.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1733e-05 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927802+0000", 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001848489 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927919+0000", 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011676900000000001 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:4361 3.8 e31/24 3:1094aab9:::rbd_data.1124ef2748ce.0000000000000003:head v 31'32, mlcod=31'29)", 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:12.927077+0000", 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.3976108260000002, 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0035751149999999998, 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927077+0000", 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927077+0000", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927079+0000", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1459999999999999e-06 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927097+0000", 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8213e-05 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927133+0000", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5429999999999998e-05 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927154+0000", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1067e-05 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927312+0000", 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015816199999999999 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.927367+0000", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5442999999999998e-05 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.930536+0000", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031685049999999998 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:12.930652+0000", 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116149 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:15.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:15.410 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:15.411 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-22T01:14:14.496271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":152369292,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":6329,"num_read_kb":62830,"num_write":755,"num_write_kb":13798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":706,"ondisk_log_size":706,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249668,"kb_used_data":20824,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754725052,"statfs":{"total":773094113280,"available":772838453248,"internally_reserved":0,"allocated":21323776,"data_stored":19443430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":128230753,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":5943,"num_read_kb":62444,"num_write":542,"num_write_kb":11032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.967651"},"pg_stats":[{"pgid":"3.1f","version":"29'9","reported_seq":350,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.589956+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:14:12.589956+0000","last_peered":"2024-09-22T01:14:12.589956+0000","last_clean":"2024-09-22T01:14:12.589956+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:14:12.589956+0000","last_undegraded":"2024-09-22T01:14:12.589956+0000","last_fullsized":"2024-09-22T01:14:12.589956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:41.772019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":6684688,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":318,"num_read_kb":7906,"num_write":9,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"29'5","reported_seq":138,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.589983+0000","last_change":"2024-09-22T01:14:06.517323+0000","last_active":"2024-09-22T01:14:12.589983+0000","last_peered":"2024-09-22T01:14:12.589983+0000","last_clean":"2024-09-22T01:14:12.589983+0000","last_became_active":"2024-09-22T01:14:04.516518+0000","last_became_peered":"2024-09-22T01:14:04.516518+0000","last_unstale":"2024-09-22T01:14:12.589983+0000","last_undegraded":"2024-09-22T01:14:12.589983+0000","last_fullsized":"2024-09-22T01:14:12.589983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:20:38.096086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":2392064,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":104,"num_read_kb":3305,"num_write":5,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"29'5","reported_seq":312,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591512+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:14:12.591512+0000","last_peered":"2024-09-22T01:14:12.591512+0000","last_clean":"2024-09-22T01:14:12.591512+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:14:12.591512+0000","last_undegraded":"2024-09-22T01:14:12.591512+0000","last_fullsized":"2024-09-22T01:14:12.591512+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:11:02.058504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":1114112,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":293,"num_read_kb":1312,"num_write":5,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"30'8","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.588989+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:14:12.588989+0000","last_peered":"2024-09-22T01:14:12.588989+0000","last_clean":"2024-09-22T01:14:12.588989+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:14:12.588989+0000","last_undegraded":"2024-09-22T01:14:12.588989+0000","last_fullsized":"2024-09-22T01:14:12.588989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:43:42.549906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"27'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.597083+0000","last_change":"2024-09-22T01:14:06.510535+0000","last_active":"2024-09-22T01:14:12.597083+0000","last_peered":"2024-09-22T01:14:12.597083+0000","last_clean":"2024-09-22T01:14:12.597083+0000","last_became_active":"2024-09-22T01:14:04.502656+0000","last_became_peered":"2024-09-22T01:14:04.502656+0000","last_unstale":"2024-09-22T01:14:12.597083+0000","last_undegraded":"2024-09-22T01:14:12.597083+0000","last_fullsized":"2024-09-22T01:14:12.597083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:49:50.564815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"27'2","reported_seq":124,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.121409+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:14:13.121409+0000","last_peered":"2024-09-22T01:14:13.121409+0000","last_clean":"2024-09-22T01:14:13.121409+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:14:13.121409+0000","last_undegraded":"2024-09-22T01:14:13.121409+0000","last_fullsized":"2024-09-22T01:14:13.121409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:19.886811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"30'45","reported_seq":613,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.596958+0000","last_change":"2024-09-22T01:14:06.509219+0000","last_active":"2024-09-22T01:14:12.596958+0000","last_peered":"2024-09-22T01:14:12.596958+0000","last_clean":"2024-09-22T01:14:12.596958+0000","last_became_active":"2024-09-22T01:14:04.511622+0000","last_became_peered":"2024-09-22T01:14:04.511622+0000","last_unstale":"2024-09-22T01:14:12.596958+0000","last_undegraded":"2024-09-22T01:14:12.596958+0000","last_fullsized":"2024-09-22T01:14:12.596958+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:32.088989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":5439488,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1081,"num_read_kb":6155,"num_write":40,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"29'5","reported_seq":340,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.589959+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:14:12.589959+0000","last_peered":"2024-09-22T01:14:12.589959+0000","last_clean":"2024-09-22T01:14:12.589959+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:14:12.589959+0000","last_undegraded":"2024-09-22T01:14:12.589959+0000","last_fullsized":"2024-09-22T01:14:12.589959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:53:48.741318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":6696619,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":308,"num_read_kb":7837,"num_write":5,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"31'22","reported_seq":64,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.012452+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:14:13.012452+0000","last_peered":"2024-09-22T01:14:13.012452+0000","last_clean":"2024-09-22T01:14:13.012452+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:14:13.012452+0000","last_undegraded":"2024-09-22T01:14:13.012452+0000","last_fullsized":"2024-09-22T01:14:13.012452+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:05.043233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":3801088,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":90,"num_write":17,"num_write_kb":1797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"31'55","reported_seq":94,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.111613+0000","last_change":"2024-09-22T01:14:10.574662+0000","last_active":"2024-09-22T01:14:13.111613+0000","last_peered":"2024-09-22T01:14:13.111613+0000","last_clean":"2024-09-22T01:14:13.111613+0000","last_became_active":"2024-09-22T01:14:10.543526+0000","last_became_peered":"2024-09-22T01:14:10.543526+0000","last_unstale":"2024-09-22T01:14:13.111613+0000","last_undegraded":"2024-09-22T01:14:13.111613+0000","last_fullsized":"2024-09-22T01:14:13.111613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:59.456774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":37,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":186,"num_write":66,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"27'4","reported_seq":31,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T01:14:11.614151+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:14:11.614151+0000","last_peered":"2024-09-22T01:14:11.614151+0000","last_clean":"2024-09-22T01:14:11.614151+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:14:11.614151+0000","last_undegraded":"2024-09-22T01:14:11.614151+0000","last_fullsized":"2024-09-22T01:14:11.614151+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:58:24.154402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"31'20","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.133758+0000","last_change":"2024-09-22T01:14:10.560133+0000","last_active":"2024-09-22T01:14:13.133758+0000","last_peered":"2024-09-22T01:14:13.133758+0000","last_clean":"2024-09-22T01:14:13.133758+0000","last_became_active":"2024-09-22T01:14:10.539067+0000","last_became_peered":"2024-09-22T01:14:10.539067+0000","last_unstale":"2024-09-22T01:14:13.133758+0000","last_undegraded":"2024-09-22T01:14:13.133758+0000","last_fullsized":"2024-09-22T01:14:13.133758+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:27:17.657169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":3932160,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":37,"num_write":9,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591501+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:14:12.591501+0000","last_peered":"2024-09-22T01:14:12.591501+0000","last_clean":"2024-09-22T01:14:12.591501+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:14:12.591501+0000","last_undegraded":"2024-09-22T01:14:12.591501+0000","last_fullsized":"2024-09-22T01:14:12.591501+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"31'48","reported_seq":312,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.155742+0000","last_change":"2024-09-22T01:14:06.510510+0000","last_active":"2024-09-22T01:14:13.155742+0000","last_peered":"2024-09-22T01:14:13.155742+0000","last_clean":"2024-09-22T01:14:13.155742+0000","last_became_active":"2024-09-22T01:14:04.523389+0000","last_became_peered":"2024-09-22T01:14:04.523389+0000","last_unstale":"2024-09-22T01:14:13.155742+0000","last_undegraded":"2024-09-22T01:14:13.155742+0000","last_fullsized":"2024-09-22T01:14:13.155742+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:22:56.348548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":638,"num_read_kb":555,"num_write":45,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.597379+0000","last_change":"2024-09-22T01:13:57.058638+0000","last_active":"2024-09-22T01:14:12.597379+0000","last_peered":"2024-09-22T01:14:12.597379+0000","last_clean":"2024-09-22T01:14:12.597379+0000","last_became_active":"2024-09-22T01:13:55.015176+0000","last_became_peered":"2024-09-22T01:13:55.015176+0000","last_unstale":"2024-09-22T01:14:12.597379+0000","last_undegraded":"2024-09-22T01:14:12.597379+0000","last_fullsized":"2024-09-22T01:14:12.597379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:07:12.063497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076979700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.597314+0000","last_change":"2024-09-22T01:14:06.510012+0000","last_active":"2024-09-22T01:14:12.597314+0000","last_peered":"2024-09-22T01:14:12.597314+0000","last_clean":"2024-09-22T01:14:12.597314+0000","last_became_active":"2024-09-22T01:14:04.529848+0000","last_became_peered":"2024-09-22T01:14:04.529848+0000","last_unstale":"2024-09-22T01:14:12.597314+0000","last_undegraded":"2024-09-22T01:14:12.597314+0000","last_fullsized":"2024-09-22T01:14:12.597314+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:20.878428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591703+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:14:12.591703+0000","last_peered":"2024-09-22T01:14:12.591703+0000","last_clean":"2024-09-22T01:14:12.591703+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:14:12.591703+0000","last_undegraded":"2024-09-22T01:14:12.591703+0000","last_fullsized":"2024-09-22T01:14:12.591703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"29'3","reported_seq":106,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.617329+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:14:12.617329+0000","last_peered":"2024-09-22T01:14:12.617329+0000","last_clean":"2024-09-22T01:14:12.617329+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:14:12.617329+0000","last_undegraded":"2024-09-22T01:14:12.617329+0000","last_fullsized":"2024-09-22T01:14:12.617329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:12:43.015136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":2390898,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":2367,"num_write":3,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.590035+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:14:12.590035+0000","last_peered":"2024-09-22T01:14:12.590035+0000","last_clean":"2024-09-22T01:14:12.590035+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:14:12.590035+0000","last_undegraded":"2024-09-22T01:14:12.590035+0000","last_fullsized":"2024-09-22T01:14:12.590035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.586833+0000","last_change":"2024-09-22T01:14:06.509901+0000","last_active":"2024-09-22T01:14:12.586833+0000","last_peered":"2024-09-22T01:14:12.586833+0000","last_clean":"2024-09-22T01:14:12.586833+0000","last_became_active":"2024-09-22T01:14:04.528349+0000","last_became_peered":"2024-09-22T01:14:04.528349+0000","last_unstale":"2024-09-22T01:14:12.586833+0000","last_undegraded":"2024-09-22T01:14:12.586833+0000","last_fullsized":"2024-09-22T01:14:12.586833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:28:26.084391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"30'10","reported_seq":38,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.470715+0000","last_change":"2024-09-22T01:14:06.521461+0000","last_active":"2024-09-22T01:14:12.470715+0000","last_peered":"2024-09-22T01:14:12.470715+0000","last_clean":"2024-09-22T01:14:12.470715+0000","last_became_active":"2024-09-22T01:14:04.507462+0000","last_became_peered":"2024-09-22T01:14:04.507462+0000","last_unstale":"2024-09-22T01:14:12.470715+0000","last_undegraded":"2024-09-22T01:14:12.470715+0000","last_fullsized":"2024-09-22T01:14:12.470715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:20:20.139447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":6160384,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":0,"num_write":11,"num_write_kb":1663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591347+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:14:12.591347+0000","last_peered":"2024-09-22T01:14:12.591347+0000","last_clean":"2024-09-22T01:14:12.591347+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:14:12.591347+0000","last_undegraded":"2024-09-22T01:14:12.591347+0000","last_fullsized":"2024-09-22T01:14:12.591347+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"30'11","reported_seq":134,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.589653+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:14:12.589653+0000","last_peered":"2024-09-22T01:14:12.589653+0000","last_clean":"2024-09-22T01:14:12.589653+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:14:12.589653+0000","last_undegraded":"2024-09-22T01:14:12.589653+0000","last_fullsized":"2024-09-22T01:14:12.589653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:57:53.135479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":2392121,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":2756,"num_write":11,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.586167+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:14:12.586167+0000","last_peered":"2024-09-22T01:14:12.586167+0000","last_clean":"2024-09-22T01:14:12.586167+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:14:12.586167+0000","last_undegraded":"2024-09-22T01:14:12.586167+0000","last_fullsized":"2024-09-22T01:14:12.586167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"30'34","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.617184+0000","last_change":"2024-09-22T01:14:06.510830+0000","last_active":"2024-09-22T01:14:12.617184+0000","last_peered":"2024-09-22T01:14:12.617184+0000","last_clean":"2024-09-22T01:14:12.617184+0000","last_became_active":"2024-09-22T01:14:04.528610+0000","last_became_peered":"2024-09-22T01:14:04.528610+0000","last_unstale":"2024-09-22T01:14:12.617184+0000","last_undegraded":"2024-09-22T01:14:12.617184+0000","last_fullsized":"2024-09-22T01:14:12.617184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:04:41.777944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":41,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":159,"num_read_kb":128,"num_write":57,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591259+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:14:12.591259+0000","last_peered":"2024-09-22T01:14:12.591259+0000","last_clean":"2024-09-22T01:14:12.591259+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:14:12.591259+0000","last_undegraded":"2024-09-22T01:14:12.591259+0000","last_fullsized":"2024-09-22T01:14:12.591259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"31'27","reported_seq":325,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.045391+0000","last_change":"2024-09-22T01:14:06.509600+0000","last_active":"2024-09-22T01:14:13.045391+0000","last_peered":"2024-09-22T01:14:13.045391+0000","last_clean":"2024-09-22T01:14:13.045391+0000","last_became_active":"2024-09-22T01:14:04.512607+0000","last_became_peered":"2024-09-22T01:14:04.512607+0000","last_unstale":"2024-09-22T01:14:13.045391+0000","last_undegraded":"2024-09-22T01:14:13.045391+0000","last_fullsized":"2024-09-22T01:14:13.045391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:34:06.950872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":19070992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":294,"num_read_kb":4537,"num_write":20,"num_write_kb":214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591395+0000","last_change":"2024-09-22T01:14:10.551838+0000","last_active":"2024-09-22T01:14:12.591395+0000","last_peered":"2024-09-22T01:14:12.591395+0000","last_clean":"2024-09-22T01:14:12.591395+0000","last_became_active":"2024-09-22T01:14:10.539121+0000","last_became_peered":"2024-09-22T01:14:10.539121+0000","last_unstale":"2024-09-22T01:14:12.591395+0000","last_undegraded":"2024-09-22T01:14:12.591395+0000","last_fullsized":"2024-09-22T01:14:12.591395+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:48.481715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":185,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.589015+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:14:12.589015+0000","last_peered":"2024-09-22T01:14:12.589015+0000","last_clean":"2024-09-22T01:14:12.589015+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:14:12.589015+0000","last_undegraded":"2024-09-22T01:14:12.589015+0000","last_fullsized":"2024-09-22T01:14:12.589015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"31'39","reported_seq":77,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.135247+0000","last_change":"2024-09-22T01:14:06.509555+0000","last_active":"2024-09-22T01:14:13.135247+0000","last_peered":"2024-09-22T01:14:13.135247+0000","last_clean":"2024-09-22T01:14:13.135247+0000","last_became_active":"2024-09-22T01:14:04.523352+0000","last_became_peered":"2024-09-22T01:14:04.523352+0000","last_unstale":"2024-09-22T01:14:13.135247+0000","last_undegraded":"2024-09-22T01:14:13.135247+0000","last_fullsized":"2024-09-22T01:14:13.135247+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:29:01.199199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":26188948,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":24,"num_write":22,"num_write_kb":340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"31'24","reported_seq":664,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.138640+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:14:13.138640+0000","last_peered":"2024-09-22T01:14:13.138640+0000","last_clean":"2024-09-22T01:14:13.138640+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:14:13.138640+0000","last_undegraded":"2024-09-22T01:14:13.138640+0000","last_fullsized":"2024-09-22T01:14:13.138640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:42:04.024512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":26569112,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":14123,"num_write":21,"num_write_kb":1752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"30'19","reported_seq":421,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591323+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:14:12.591323+0000","last_peered":"2024-09-22T01:14:12.591323+0000","last_clean":"2024-09-22T01:14:12.591323+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:14:12.591323+0000","last_undegraded":"2024-09-22T01:14:12.591323+0000","last_fullsized":"2024-09-22T01:14:12.591323+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:06.477323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":1114112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":573,"num_read_kb":1548,"num_write":17,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"28'3","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.586182+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:14:12.586182+0000","last_peered":"2024-09-22T01:14:12.586182+0000","last_clean":"2024-09-22T01:14:12.586182+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:14:12.586182+0000","last_undegraded":"2024-09-22T01:14:12.586182+0000","last_fullsized":"2024-09-22T01:14:12.586182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:13.278376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":5,"num_write":3,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"29'5","reported_seq":361,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.085563+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:14:12.085563+0000","last_peered":"2024-09-22T01:14:12.085563+0000","last_clean":"2024-09-22T01:14:12.085563+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:14:12.085563+0000","last_undegraded":"2024-09-22T01:14:12.085563+0000","last_fullsized":"2024-09-22T01:14:12.085563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:24:11.560230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":3433833,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":330,"num_read_kb":3386,"num_write":5,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"27'4","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.591614+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:14:12.591614+0000","last_peered":"2024-09-22T01:14:12.591614+0000","last_clean":"2024-09-22T01:14:12.591614+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:14:12.591614+0000","last_undegraded":"2024-09-22T01:14:12.591614+0000","last_fullsized":"2024-09-22T01:14:12.591614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:25:46.575069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"29'89","reported_seq":389,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.587089+0000","last_change":"2024-09-22T01:14:06.511183+0000","last_active":"2024-09-22T01:14:12.587089+0000","last_peered":"2024-09-22T01:14:12.587089+0000","last_clean":"2024-09-22T01:14:12.587089+0000","last_became_active":"2024-09-22T01:14:04.512742+0000","last_became_peered":"2024-09-22T01:14:04.512742+0000","last_unstale":"2024-09-22T01:14:12.587089+0000","last_undegraded":"2024-09-22T01:14:12.587089+0000","last_fullsized":"2024-09-22T01:14:12.587089+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:30:24.838721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":983078,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":678,"num_read_kb":1378,"num_write":139,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"30'10","reported_seq":42,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.492409+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:14:12.492409+0000","last_peered":"2024-09-22T01:14:12.492409+0000","last_clean":"2024-09-22T01:14:12.492409+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:14:12.492409+0000","last_undegraded":"2024-09-22T01:14:12.492409+0000","last_fullsized":"2024-09-22T01:14:12.492409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:30.429596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":1671168,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":9,"num_write_kb":1605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.588927+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:14:12.588927+0000","last_peered":"2024-09-22T01:14:12.588927+0000","last_clean":"2024-09-22T01:14:12.588927+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:14:12.588927+0000","last_undegraded":"2024-09-22T01:14:12.588927+0000","last_fullsized":"2024-09-22T01:14:12.588927+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:06:15.176069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"31'35","reported_seq":60,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.135129+0000","last_change":"2024-09-22T01:14:10.582207+0000","last_active":"2024-09-22T01:14:13.135129+0000","last_peered":"2024-09-22T01:14:13.135129+0000","last_clean":"2024-09-22T01:14:13.135129+0000","last_became_active":"2024-09-22T01:14:10.539572+0000","last_became_peered":"2024-09-22T01:14:10.539572+0000","last_unstale":"2024-09-22T01:14:13.135129+0000","last_undegraded":"2024-09-22T01:14:13.135129+0000","last_fullsized":"2024-09-22T01:14:13.135129+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:47:10.108587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":8019873,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":14,"num_write":20,"num_write_kb":1752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"29'11","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:12.617241+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:14:12.617241+0000","last_peered":"2024-09-22T01:14:12.617241+0000","last_clean":"2024-09-22T01:14:12.617241+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:14:12.617241+0000","last_undegraded":"2024-09-22T01:14:12.617241+0000","last_fullsized":"2024-09-22T01:14:12.617241+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:20.185012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":63,"num_write":11,"num_write_kb":182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"31'30","reported_seq":203,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-22T01:14:13.134203+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:14:13.134203+0000","last_peered":"2024-09-22T01:14:13.134203+0000","last_clean":"2024-09-22T01:14:13.134203+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:14:13.134203+0000","last_undegraded":"2024-09-22T01:14:13.134203+0000","last_fullsized":"2024-09-22T01:14:13.134203+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:40:49.860692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":23724032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":175,"num_read_kb":4909,"num_write":22,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":151778905,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6213,"num_read_kb":62730,"num_write":594,"num_write_kb":11380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17289216,"data_stored":17064121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":586,"ondisk_log_size":586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":417608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":385024,"data_stored":167520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29552,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342288,"statfs":{"total":96636764160,"available":96606502912,"internally_reserved":0,"allocated":2580480,"data_stored":2348275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29372,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342468,"statfs":{"total":96636764160,"available":96606687232,"internally_reserved":0,"allocated":2400256,"data_stored":2150467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":5216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":5341184,"data_stored":5108736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34808,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337032,"statfs":{"total":96636764160,"available":96601120768,"internally_reserved":0,"allocated":3584000,"data_stored":3346698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33768,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338072,"statfs":{"total":96636764160,"available":96602185728,"internally_reserved":0,"allocated":2637824,"data_stored":2402860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34844,"kb_used_data":3644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336996,"statfs":{"total":96636764160,"available":96601083904,"internally_reserved":0,"allocated":3731456,"data_stored":3501266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3252224,"data_stored":3236888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1712128,"data_stored":1683928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3207168,"data_stored":3179734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4997120,"data_stored":4973379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2084864,"data_stored":2032868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1654784,"data_stored":1629343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":331776,"data_stored":289044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:15.413 INFO:tasks.thrashosds.thrasher:Setting 2.6 to [1, 0] 2024-09-22T01:14:15.413 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.6', '1', '0'] 2024-09-22T01:14:15.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.6 1 0 2024-09-22T01:14:15.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:15.455 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:15.455+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:15.518 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:15.515+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:15.577 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:15.577 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:15.577 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:15.590 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:14:15.593 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:15.619 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:15.617+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:15.720 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:15.719+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:15.806 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:15.806 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:15.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:15.867 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:15.867+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:15.890 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-22T01:14:15.921 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:15.921+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:15.937 INFO:teuthology.orchestra.run.smithi096.stdout:471 punch from 0x5fe11d5 to 0x5fe89cc, (0x77f7 bytes) 2024-09-22T01:14:15.944 INFO:teuthology.orchestra.run.smithi096.stdout:472 trunc from 0x8cd484a to 0x7fa7afa 2024-09-22T01:14:15.976 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:15.977 INFO:teuthology.orchestra.run.smithi096.stdout:475 write 0x511f4e0 thru 0x512413d (0x4c5e bytes) 2024-09-22T01:14:15.990 INFO:teuthology.orchestra.run.smithi096.stdout:477 read 0x126b46b thru 0x126e09c (0x2c32 bytes) 2024-09-22T01:14:15.997 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1448 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 5341184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.983137+0000", 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.014483899, 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096600439999999996, 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:15.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1448 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983137+0000", 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:15.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983137+0000", 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983137+0000", 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983139+0000", 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.968e-06 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983146+0000", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2480000000000001e-06 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.983150+0000", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3769999999999998e-06 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.988846+0000", 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056955570000000004 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.988955+0000", 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010907 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.992797+0000", 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038424739999999998 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1725 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984829+0000", 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0127920189999999, 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090553130000000006, 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1725 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984829+0000", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984829+0000", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984829+0000", 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:14:16.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984836+0000", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6159999999999998e-06 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984843+0000", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984848+0000", 2024-09-22T01:14:16.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993727+0000", 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088793110000000008 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993810+0000", 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993884+0000", 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4763000000000002e-05 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1726 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 4358144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984869+0000", 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0127519209999996, 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092055769999999995, 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1726 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984869+0000", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984869+0000", 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984870+0000", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999998e-07 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984872+0000", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4889999999999998e-06 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984879+0000", 2024-09-22T01:14:16.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3100000000000003e-06 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984891+0000", 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2115e-05 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993915+0000", 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090240370000000004 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993998+0000", 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2142000000000001e-05 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994075+0000", 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7040999999999996e-05 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1727 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 4685824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984899+0000", 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0127219629999997, 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009413856, 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1727 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984899+0000", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984899+0000", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984900+0000", 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984902+0000", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0679999999999999e-06 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984908+0000", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984913+0000", 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-06 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994128+0000", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092155129999999998 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994216+0000", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7849999999999994e-05 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994313+0000", 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6841000000000003e-05 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1728 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 4915200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984919+0000", 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0127023020000001, 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095908759999999999, 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1728 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984919+0000", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984919+0000", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984919+0000", 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0699999999999998e-07 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984921+0000", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7969999999999999e-06 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984927+0000", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2360000000000001e-06 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984932+0000", 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9570000000000001e-06 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994350+0000", 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009417712 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994422+0000", 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2727999999999999e-05 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994510+0000", 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7138999999999994e-05 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1729 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984939+0000", 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0126815809999998, 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097377680000000008, 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1729 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984939+0000", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984939+0000", 2024-09-22T01:14:16.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984940+0000", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984942+0000", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984952+0000", 2024-09-22T01:14:16.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984957+0000", 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9230000000000001e-06 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994545+0000", 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095873250000000007 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994612+0000", 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7848999999999994e-05 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994677+0000", 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4769000000000003e-05 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1730 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984964+0000", 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0126569910000001, 2024-09-22T01:14:16.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098632230000000008, 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1730 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984964+0000", 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984964+0000", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984964+0000", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984966+0000", 2024-09-22T01:14:16.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.827e-06 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984972+0000", 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0769999999999996e-06 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984977+0000", 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994706+0000", 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0097296369999999993 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994760+0000", 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4132000000000003e-05 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994827+0000", 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6802000000000002e-05 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1731 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.984984+0000", 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0126370900000001, 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0099909460000000005, 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1731 2024-09-22T01:14:16.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984984+0000", 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984984+0000", 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984984+0000", 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984986+0000", 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.88e-06 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984993+0000", 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3749999999999999e-06 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.984997+0000", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6630000000000001e-06 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994855+0000", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0098579670000000005 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994923+0000", 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8045000000000003e-05 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.994975+0000", 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1659999999999997e-05 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1732 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985004+0000", 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0126170969999997, 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010122435000000001, 2024-09-22T01:14:16.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1732 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985004+0000", 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985004+0000", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985004+0000", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.27e-07 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985006+0000", 2024-09-22T01:14:16.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.973e-06 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985013+0000", 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6510000000000004e-06 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985018+0000", 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995002+0000", 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0099848449999999991 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995069+0000", 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7056000000000003e-05 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995126+0000", 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6925000000000001e-05 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1733 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985024+0000", 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0125965270000004, 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010239345, 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1733 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985024+0000", 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985024+0000", 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985025+0000", 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985027+0000", 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.894e-06 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985033+0000", 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5740000000000001e-06 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985039+0000", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0129999999999999e-06 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995155+0000", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010115509 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995218+0000", 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3170999999999995e-05 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995264+0000", 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5841999999999999e-05 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1734 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985046+0000", 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.012574839, 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010366558999999999, 2024-09-22T01:14:16.042 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1734 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985046+0000", 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985046+0000", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985047+0000", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6800000000000001e-07 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985049+0000", 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.136e-06 2024-09-22T01:14:16.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985061+0000", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2021000000000001e-05 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985066+0000", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995292+0000", 2024-09-22T01:14:16.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010226094 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995359+0000", 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7052999999999997e-05 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995413+0000", 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3474000000000003e-05 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1735 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985074+0000", 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.012547165, 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010493255999999999, 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1735 2024-09-22T01:14:16.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985074+0000", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985074+0000", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985074+0000", 2024-09-22T01:14:16.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0100000000000005e-07 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985083+0000", 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0140000000000004e-06 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985091+0000", 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1729999999999998e-06 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985101+0000", 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0251999999999999e-05 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995445+0000", 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010343991 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995503+0000", 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000003e-05 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995567+0000", 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3825000000000006e-05 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1736 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985109+0000", 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0125121330000004, 2024-09-22T01:14:16.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010582734999999999, 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1736 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985109+0000", 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985109+0000", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985109+0000", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9400000000000005e-07 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985112+0000", 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4250000000000001e-06 2024-09-22T01:14:16.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985120+0000", 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4859999999999997e-06 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985126+0000", 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2320000000000001e-06 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995597+0000", 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010471561000000001 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995638+0000", 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1208999999999998e-05 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995692+0000", 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3227999999999999e-05 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1737 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.985132+0000", 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0124891180000004, 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010696004, 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1737 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.056 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985132+0000", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985132+0000", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985132+0000", 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2024-09-22T01:14:16.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985138+0000", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3079999999999998e-06 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985145+0000", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.888e-06 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.985150+0000", 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1830000000000002e-06 2024-09-22T01:14:16.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995718+0000", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010567953 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995758+0000", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9795000000000001e-05 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.995828+0000", 2024-09-22T01:14:16.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0392e-05 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2012 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.993348+0000", 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0042730930000001, 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090493799999999992, 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2012 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993348+0000", 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993348+0000", 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993348+0000", 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993350+0000", 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.618e-06 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993354+0000", 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-06 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993363+0000", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0939999999999997e-06 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002250+0000", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0088864619999999995 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002329+0000", 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9587999999999998e-05 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002397+0000", 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8193000000000003e-05 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2013 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 7143424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.993368+0000", 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0042534219999997, 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092182140000000006, 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2013 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993368+0000", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993368+0000", 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993368+0000", 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.96e-07 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993369+0000", 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.429e-06 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993373+0000", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7100000000000001e-06 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993376+0000", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7420000000000002e-06 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002428+0000", 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0090523800000000005 2024-09-22T01:14:16.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002507+0000", 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8856000000000005e-05 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002586+0000", 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8701000000000004e-05 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2014 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 7471104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.993380+0000", 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.004240759, 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093818809999999999, 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2014 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993380+0000", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993380+0000", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993380+0000", 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4299999999999999e-07 2024-09-22T01:14:16.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993382+0000", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2729999999999999e-06 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993393+0000", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1654999999999999e-05 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993401+0000", 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-22T01:14:16.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002615+0000", 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0092147110000000004 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002692+0000", 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6366999999999996e-05 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002762+0000", 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0400000000000004e-05 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2015 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 8028160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.993411+0000", 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0042100449999998, 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095264630000000006, 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2015 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993411+0000", 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993411+0000", 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993411+0000", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8100000000000003e-07 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993414+0000", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5600000000000001e-06 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993419+0000", 2024-09-22T01:14:16.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.361e-06 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993423+0000", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3589999999999999e-06 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002792+0000", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093688399999999998 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002867+0000", 2024-09-22T01:14:16.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5853999999999996e-05 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002937+0000", 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0007999999999998e-05 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2016 3.c 3:364111de:::rbd_data.1124a8bc99f2.0000000000000001:4 [read 8355840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:11.993427+0000", 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0041940819999997, 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.009688482, 2024-09-22T01:14:16.077 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2016 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993427+0000", 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993427+0000", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993427+0000", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.41e-07 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993429+0000", 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.341e-06 2024-09-22T01:14:16.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993432+0000", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6990000000000001e-06 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:11.993435+0000", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.768e-06 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.002968+0000", 2024-09-22T01:14:16.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0095331860000000008 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.003044+0000", 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5811999999999999e-05 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.003115+0000", 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1434999999999996e-05 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3933 3.f 3:f087e1b2:::rbd_data.1124ef2748ce.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.119953+0000", 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.877668468, 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0093648949999999998, 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3933 2024-09-22T01:14:16.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.119953+0000", 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.119953+0000", 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.119955+0000", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121051+0000", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001095384 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121076+0000", 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5364e-05 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121087+0000", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0815000000000001e-05 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121166+0000", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9171999999999993e-05 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121300+0000", 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013444100000000001 2024-09-22T01:14:16.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.121614+0000", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031337200000000002 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.127422+0000", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058084499999999997 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.129195+0000", 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001773022 2024-09-22T01:14:16.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.129265+0000", 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9941e-05 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.129317+0000", 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2339999999999997e-05 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.088 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:16.088 INFO:teuthology.orchestra.run.smithi096.stdout:478 write 0xaa5e030 thru 0xaa6bc1d (0xdbee bytes) 2024-09-22T01:14:16.089 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:16.090 INFO:teuthology.orchestra.run.smithi096.stdout:479 read 0x8af71a9 thru 0x8b02df4 (0xbc4c bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:482 writesame 0x4326d64 thru 0x432a8b8 data_size 0x3(0x3b55 bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:483 punch from 0x2bd729b to 0x2bdec19, (0x797e bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:484 read 0x5435664 thru 0x5436a6b (0x1408 bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:486 writesame 0x175093e thru 0x175b0e0 data_size 0x5(0xa7a3 bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:487 punch from 0x566ae7d to 0x566ff88, (0x510b bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:489 write 0x7f21bbc thru 0x7f21e3d (0x282 bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:490 read 0x73f1c47 thru 0x74004cf (0xe889 bytes) 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:491 trunc from 0xaa6bc1e to 0x95b0ff0 2024-09-22T01:14:16.091 INFO:teuthology.orchestra.run.smithi096.stdout:492 write 0x2cf725b thru 0x2d065dd (0xf383 bytes) 2024-09-22T01:14:16.091 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:16.087+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:16.094 INFO:teuthology.orchestra.run.smithi096.stdout:493 write 0x5d1cb09 thru 0x5d2013d (0x3635 bytes) 2024-09-22T01:14:16.100 INFO:teuthology.orchestra.run.smithi096.stdout:495 read 0x4d3232c thru 0x4d36312 (0x3fe7 bytes) 2024-09-22T01:14:16.100 INFO:teuthology.orchestra.run.smithi096.stdout:496 write 0x73edc9e thru 0x73f923f (0xb5a2 bytes) 2024-09-22T01:14:16.112 INFO:teuthology.orchestra.run.smithi096.stdout:497 read 0x5234f4b thru 0x523fb47 (0xabfd bytes) 2024-09-22T01:14:16.112 INFO:teuthology.orchestra.run.smithi096.stdout:498 write 0x1fca77b thru 0x1fda21e (0xfaa4 bytes) 2024-09-22T01:14:16.119 INFO:teuthology.orchestra.run.smithi096.stdout:499 read 0x2005de6 thru 0x2014b8b (0xeda6 bytes) 2024-09-22T01:14:16.119 INFO:teuthology.orchestra.run.smithi096.stdout:502 read 0xb09fd4 thru 0xb0cc41 (0x2c6e bytes) 2024-09-22T01:14:16.120 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.120 INFO:teuthology.orchestra.run.smithi096.stdout:507 read 0x2db8c72 thru 0x2dc3ef9 (0xb288 bytes) 2024-09-22T01:14:16.121 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.141 INFO:teuthology.orchestra.run.smithi096.stdout:509 write 0xe1535e1 thru 0xe1551e4 (0x1c04 bytes) 2024-09-22T01:14:16.147 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.148 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.148 INFO:teuthology.orchestra.run.smithi096.stdout:513 writesame 0x7f47a41 thru 0x7f55eaa data_size 0x1a(0xe46a bytes) 2024-09-22T01:14:16.153 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:516 read 0xb8913dd thru 0xb892cf3 (0x1917 bytes) 2024-09-22T01:14:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:518 punch from 0x7b1cb88 to 0x7b2bab1, (0xef29 bytes) 2024-09-22T01:14:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:520 read 0x14f2f25 thru 0x14ffe97 (0xcf73 bytes) 2024-09-22T01:14:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:523 writesame 0x9881752 thru 0x9885624 data_size 0x3(0x3ed3 bytes) 2024-09-22T01:14:16.160 INFO:teuthology.orchestra.run.smithi096.stdout:524 punch from 0xc9c988e to 0xc9d04ad, (0x6c1f bytes) 2024-09-22T01:14:16.160 INFO:teuthology.orchestra.run.smithi096.stdout:526 read 0xc659357 thru 0xc66229c (0x8f46 bytes) 2024-09-22T01:14:16.160 INFO:teuthology.orchestra.run.smithi096.stdout:527 writesame 0x131f0f6 thru 0x1327097 data_size 0x22(0x7fa2 bytes) 2024-09-22T01:14:16.177 INFO:teuthology.orchestra.run.smithi096.stdout:528 write 0xb5c9498 thru 0xb5d3a93 (0xa5fc bytes) 2024-09-22T01:14:16.186 INFO:teuthology.orchestra.run.smithi096.stdout:529 trunc from 0xe1551e5 to 0xec26708 2024-09-22T01:14:16.189 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:16.189+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:16.204 INFO:teuthology.orchestra.run.smithi096.stdout:530 trunc from 0xec26708 to 0x9b50e99 2024-09-22T01:14:16.226 INFO:teuthology.orchestra.run.smithi096.stdout:531 read 0x7bfb214 thru 0x7c00d2d (0x5b1a bytes) 2024-09-22T01:14:16.232 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:16.232 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:16.232 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:16.242 INFO:teuthology.orchestra.run.smithi096.stdout:532 write 0xbeea687 thru 0xbef748e (0xce08 bytes) 2024-09-22T01:14:16.247 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:16.250 INFO:teuthology.orchestra.run.smithi096.stdout:533 punch from 0x5da330d to 0x5db0c45, (0xd938 bytes) 2024-09-22T01:14:16.250 INFO:teuthology.orchestra.run.smithi096.stdout:534 write 0x23f544f thru 0x23fe7cf (0x9381 bytes) 2024-09-22T01:14:16.253 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.253 INFO:teuthology.orchestra.run.smithi096.stdout:536 punch from 0x8ae57ce to 0x8aea2dd, (0x4b0f bytes) 2024-09-22T01:14:16.253 INFO:teuthology.orchestra.run.smithi096.stdout:538 write 0x64cba05 thru 0x64d83ed (0xc9e9 bytes) 2024-09-22T01:14:16.284 INFO:teuthology.orchestra.run.smithi096.stdout:539 trunc from 0xbef748f to 0x10ce3e 2024-09-22T01:14:16.290 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:16.287+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:16.351 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:16.352 INFO:teuthology.orchestra.run.smithi096.stdout:541 trunc from 0x10ce3e to 0xe80f2b 2024-09-22T01:14:16.362 INFO:teuthology.orchestra.run.smithi096.stdout:542 read 0xadd60b thru 0xae8290 (0xac86 bytes) 2024-09-22T01:14:16.362 INFO:teuthology.orchestra.run.smithi096.stdout:543 read 0x77ebed thru 0x77f2c5 (0x6d9 bytes) 2024-09-22T01:14:16.363 INFO:teuthology.orchestra.run.smithi096.stdout:545 read 0x4a2205 thru 0x4b11a1 (0xef9d bytes) 2024-09-22T01:14:16.364 INFO:teuthology.orchestra.run.smithi096.stdout:546 clone 6 order 18 su 131072 sc 5 2024-09-22T01:14:16.390 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:16.390 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:16.390 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:16.390 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:16.390 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:16.391 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:16.387+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:16.404 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:16.492 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:16.491+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:16.573 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:16.573 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:16.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.585636746999999, 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:16.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.335403789000001, 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011381442, 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234702+0000", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234713+0000", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0507e-05 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234735+0000", 2024-09-22T01:14:16.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1800000000000001e-05 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234748+0000", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3583e-05 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243350+0000", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086011609999999995 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243483+0000", 2024-09-22T01:14:16.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308499999999999 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243702+0000", 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021935499999999999 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245600+0000", 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018977219999999999 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245927+0000", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032715799999999997 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246017+0000", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9752000000000004e-05 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246082+0000", 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5813000000000002e-05 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.335327502999998, 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013805090000000001, 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234779+0000", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-06 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234789+0000", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.042e-05 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234808+0000", 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9256e-05 2024-09-22T01:14:16.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234826+0000", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7825999999999998e-05 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244277+0000", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094506409999999992 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244411+0000", 2024-09-22T01:14:16.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134513 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244661+0000", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024963399999999999 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245702+0000", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001040736 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248290+0000", 2024-09-22T01:14:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025886429999999998 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248450+0000", 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015967800000000001 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248582+0000", 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132316 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.063263534000001, 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011449948999999999, 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506842+0000", 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506918+0000", 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5715000000000003e-05 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506925+0000", 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4599999999999997e-06 2024-09-22T01:14:16.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506929+0000", 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2520000000000001e-06 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510441+0000", 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035115350000000001 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510484+0000", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510560+0000", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6184000000000001e-05 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513176+0000", 2024-09-22T01:14:16.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026151540000000002 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518234+0000", 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050580019999999998 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518267+0000", 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3800999999999997e-05 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518291+0000", 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3855000000000001e-05 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.063166532, 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011448767, 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:16.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506939+0000", 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506942+0000", 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8420000000000001e-06 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506945+0000", 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9349999999999996e-06 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506949+0000", 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511010+0000", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040611680000000004 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511054+0000", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4255999999999998e-05 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511140+0000", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5841999999999996e-05 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513889+0000", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027490869999999999 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518324+0000", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044343489999999998 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518358+0000", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4737999999999998e-05 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518387+0000", 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8572999999999999e-05 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.063093464000001, 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446914, 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507012+0000", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507015+0000", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507020+0000", 2024-09-22T01:14:16.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507023+0000", 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511408+0000", 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043849700000000002 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511451+0000", 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2735999999999999e-05 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511539+0000", 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7766e-05 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513894+0000", 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023553100000000002 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519395+0000", 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055011189999999996 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519431+0000", 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5457999999999998e-05 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519458+0000", 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7682e-05 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.062890685999999, 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015085121999999999, 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:16.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507215+0000", 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507301+0000", 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6791000000000003e-05 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507310+0000", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.653e-06 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507323+0000", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2648e-05 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511896+0000", 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045732380000000003 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511954+0000", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8319000000000002e-05 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512028+0000", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513897+0000", 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001869587 2024-09-22T01:14:16.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522235+0000", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083377590000000001 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522264+0000", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9431999999999999e-05 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522299+0000", 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4860999999999997e-05 2024-09-22T01:14:16.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.062726755, 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01505534, 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507379+0000", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507382+0000", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.258e-06 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507388+0000", 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.263e-06 2024-09-22T01:14:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507392+0000", 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.771e-06 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512533+0000", 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051413939999999997 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512618+0000", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4338000000000005e-05 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512742+0000", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012398199999999999 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513903+0000", 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001161147 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522355+0000", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084519360000000002 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522399+0000", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4508000000000001e-05 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522433+0000", 2024-09-22T01:14:16.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4177000000000002e-05 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.062644639999998, 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015058092, 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507461+0000", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507463+0000", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507475+0000", 2024-09-22T01:14:16.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.146e-05 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507479+0000", 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1049999999999997e-06 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513231+0000", 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057521810000000003 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513296+0000", 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4764999999999994e-05 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513388+0000", 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1847000000000003e-05 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.516279+0000", 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028914119999999999 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522465+0000", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006186352 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522496+0000", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0988000000000002e-05 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522518+0000", 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1792000000000001e-05 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.629 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.595423713999999, 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01429044, 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 131 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974682+0000", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974824+0000", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014190599999999999 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974839+0000", 2024-09-22T01:14:16.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974849+0000", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8369999999999992e-06 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977235+0000", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023857219999999998 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977380+0000", 2024-09-22T01:14:16.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014566199999999999 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977640+0000", 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025971799999999998 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.980903+0000", 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032634040000000001 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988830+0000", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260430000000007 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988904+0000", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4016000000000004e-05 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988971+0000", 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7929000000000006e-05 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.595235904999999, 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014424996000000001, 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132 2024-09-22T01:14:16.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974870+0000", 2024-09-22T01:14:16.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974876+0000", 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974889+0000", 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2670000000000001e-05 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974896+0000", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9370000000000004e-06 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979031+0000", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041348749999999997 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979163+0000", 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132142 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979423+0000", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025945100000000001 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981009+0000", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001585801 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989179+0000", 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081698940000000005 2024-09-22T01:14:16.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989243+0000", 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4480000000000006e-05 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989294+0000", 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0742999999999999e-05 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.595159886000001, 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484576000000001, 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 133 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974946+0000", 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0499999999999999e-06 2024-09-22T01:14:16.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974952+0000", 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974964+0000", 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2164000000000001e-05 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974976+0000", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1875999999999999e-05 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981087+0000", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061111660000000003 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981224+0000", 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013626900000000001 2024-09-22T01:14:16.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981437+0000", 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021382899999999999 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985221+0000", 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037832999999999999 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989306+0000", 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004085506 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989379+0000", 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2909000000000003e-05 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989429+0000", 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0377000000000002e-05 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.59505665, 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018165239999999999, 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 134 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975049+0000", 2024-09-22T01:14:16.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975179+0000", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129722 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975197+0000", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8113000000000001e-05 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975207+0000", 2024-09-22T01:14:16.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5100000000000004e-06 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982219+0000", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070121760000000002 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982352+0000", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133249 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982617+0000", 2024-09-22T01:14:16.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026466800000000002 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989070+0000", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064531299999999996 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.992931+0000", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003860774 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993092+0000", 2024-09-22T01:14:16.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016092499999999999 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993213+0000", 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121836 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594880377999999, 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018294939, 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 135 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975225+0000", 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975231+0000", 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7699999999999998e-06 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975243+0000", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1644000000000001e-05 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975250+0000", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.503e-06 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983862+0000", 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086120459999999999 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983989+0000", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012705299999999999 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.984185+0000", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019606499999999999 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989159+0000", 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049732120000000003 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993238+0000", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040793449999999998 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993372+0000", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013405200000000001 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993519+0000", 2024-09-22T01:14:16.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733899999999999 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594839145000002, 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018936457, 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 136 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8400000000000004e-07 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975272+0000", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975284+0000", 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2393e-05 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975292+0000", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.763e-06 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985342+0000", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010050026 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985551+0000", 2024-09-22T01:14:16.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208755 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985988+0000", 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043766699999999998 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989513+0000", 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003525233 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994036+0000", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045223279999999999 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994117+0000", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1549000000000001e-05 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994202+0000", 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4735999999999996e-05 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594783309, 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019072927, 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-22T01:14:16.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975323+0000", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975483+0000", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160416 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975500+0000", 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7127000000000001e-05 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975509+0000", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4399999999999994e-06 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987086+0000", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011576606999999999 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987414+0000", 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032788700000000001 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987806+0000", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039176100000000002 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993673+0000", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058668970000000003 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994227+0000", 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055439900000000002 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994330+0000", 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010278 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994394+0000", 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4542999999999995e-05 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594577826999998, 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019163698999999999, 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975528+0000", 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.017e-06 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975534+0000", 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975547+0000", 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2806000000000001e-05 2024-09-22T01:14:16.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975554+0000", 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989583+0000", 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014028175 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989829+0000", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024677100000000001 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.990235+0000", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040547400000000001 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994418+0000", 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041833579999999999 2024-09-22T01:14:16.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994534+0000", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011541500000000001 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994623+0000", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9703000000000005e-05 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994691+0000", 2024-09-22T01:14:16.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7366000000000005e-05 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594538177, 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021194425999999999, 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975577+0000", 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4669999999999993e-06 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975587+0000", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975594+0000", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1770000000000001e-06 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991392+0000", 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015798764 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991632+0000", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023988199999999999 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991978+0000", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034544099999999999 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996441+0000", 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004463608 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996541+0000", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9623000000000005e-05 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996644+0000", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103416 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996761+0000", 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116532 2024-09-22T01:14:16.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594499386999999, 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022802374, 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975606+0000", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0299999999999998e-07 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975744+0000", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013788599999999999 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975767+0000", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3003e-05 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975781+0000", 2024-09-22T01:14:16.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.383e-05 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994785+0000", 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019003995999999999 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994930+0000", 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144683 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.995124+0000", 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019457099999999999 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998089+0000", 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029647440000000001 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998221+0000", 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178199999999999 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998301+0000", 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9839999999999995e-05 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998408+0000", 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107236 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:16.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.594307985, 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024814201000000001, 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:16.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975798+0000", 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:16.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975807+0000", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.533e-06 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975817+0000", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9599999999999995e-06 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975824+0000", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3289999999999996e-06 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996882+0000", 2024-09-22T01:14:16.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021058038000000001 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997106+0000", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022396800000000001 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997464+0000", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035745699999999999 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998789+0000", 2024-09-22T01:14:16.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001325384 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000443+0000", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001653493 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000521+0000", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000611+0000", 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9662999999999999e-05 2024-09-22T01:14:16.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:16.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:16.694 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:16.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:16.696 INFO:teuthology.orchestra.run.smithi096.stderr:set 2.6 pg_upmap mapping to [1,0] 2024-09-22T01:14:16.697 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:16.693+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:16.741 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone5 from 0xe80f2b (overlap 0x10ce3e) to 0xd71cb 2024-09-22T01:14:16.796 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:16.795+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:16.851 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-22T01:14:16.872 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:16.872 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:16.872 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:16.890 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:16.896 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:16.897+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:16.998 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:16.997+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:17.055 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:17.055 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:17.055 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:17.070 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:17.119 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:17.119+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:17.177 INFO:teuthology.orchestra.run.smithi096.stdout:549 write 0x795c4a thru 0x7a4a5a (0xee11 bytes) 2024-09-22T01:14:17.195 INFO:teuthology.orchestra.run.smithi096.stdout:550 punch from 0x54f439 to 0x5536bd, (0x4284 bytes) 2024-09-22T01:14:17.195 INFO:teuthology.orchestra.run.smithi096.stdout:551 read 0x5e4bff thru 0x5eb42f (0x6831 bytes) 2024-09-22T01:14:17.199 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:17.199+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1738 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 131072~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1991342930000002, 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014237117000000001, 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1738 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006357+0000", 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5000999999999999e-05 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006366+0000", 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006372+0000", 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8379999999999998e-06 2024-09-22T01:14:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013432+0000", 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070601079999999998 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013499+0000", 2024-09-22T01:14:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7531000000000001e-05 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020579+0000", 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070795700000000003 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2025 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2916352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1984434369999999, 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013872165000000001, 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2025 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:17.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007035+0000", 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9309999999999998e-06 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007042+0000", 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8369999999999999e-06 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007046+0000", 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020772+0000", 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013725342 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020840+0000", 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8576000000000002e-05 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020905+0000", 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4469999999999998e-05 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2026 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1984229690000001, 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014009057, 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2026 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007056+0000", 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007062+0000", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5570000000000001e-06 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007067+0000", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8029999999999996e-06 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020933+0000", 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013865475 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020999+0000", 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390999999999995e-05 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021062+0000", 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2027 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1984031740000001, 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014158201, 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2027 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007074+0000", 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007083+0000", 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9889999999999998e-06 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007089+0000", 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5259999999999999e-06 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007094+0000", 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6530000000000002e-06 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021090+0000", 2024-09-22T01:14:17.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01399624 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021157+0000", 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6538999999999994e-05 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021231+0000", 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4640000000000004e-05 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2028 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3801088~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1983756579999998, 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014289873, 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2028 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007101+0000", 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:17.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007103+0000", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007111+0000", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007122+0000", 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988000000000001e-05 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021258+0000", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014136319 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021326+0000", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7642000000000001e-05 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021390+0000", 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4645000000000002e-05 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.233 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1983475840000004, 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:17.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:17.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.198326915, 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:17.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1982966020000001, 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1982773480000004, 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1982592460000001, 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:17.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1982385229999997, 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:17.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1982180339999999, 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1981965920000004, 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:17.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1981657449999998, 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:17.278 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.198128219, 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:17.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:17.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.198096091, 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:17.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:17.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1980730839999998, 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:17.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1587592510000002, 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:17.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:17.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1574720709999999, 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.301 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1571278879999998, 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:17.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.307 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:17.308 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:17.308 INFO:teuthology.orchestra.run.smithi096.stdout:553 write 0xaa9923a thru 0xaa9b9cd (0x2794 bytes) 2024-09-22T01:14:17.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:554 write 0x59f4b40 thru 0x59f6e12 (0x22d3 bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:555 write 0x760d0ff thru 0x7617584 (0xa486 bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:556 write 0x9876bf8 thru 0x987bc25 (0x502e bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:558 read 0x2a12998 thru 0x2a1a3da (0x7a43 bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:559 read 0x527ab28 thru 0x5288916 (0xddef bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:560 writesame 0x11de120 thru 0x11e87e4 data_size 0x39(0xa6c5 bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:561 punch from 0x29f783 to 0x2a0fd0, (0x184d bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:562 read 0x8ad2fcc thru 0x8ad5ba6 (0x2bdb bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:564 writesame 0xbb6d292 thru 0xbb76811 data_size 0x34(0x9580 bytes) 2024-09-22T01:14:17.311 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:566 trunc from 0xbb76812 to 0x505dc02 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:567 punch from 0x43557a0 to 0x43613de, (0xbc3e bytes) 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:568 read 0x1f200c0 thru 0x1f23578 (0x34b9 bytes) 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:569 write 0x4efa53c thru 0x4eff2c5 (0x4d8a bytes) 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:570 read 0xdeee5c thru 0xdfbdd6 (0xcf7b bytes) 2024-09-22T01:14:17.312 INFO:teuthology.orchestra.run.smithi096.stdout:571 read 0x15ed9d7 thru 0x15fb0d7 (0xd701 bytes) 2024-09-22T01:14:17.312 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:17.307+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:17.331 INFO:teuthology.orchestra.run.smithi096.stdout:572 write 0x95610a7 thru 0x9567251 (0x61ab bytes) 2024-09-22T01:14:17.363 INFO:teuthology.orchestra.run.smithi096.stdout:573 write 0xeaafd8d thru 0xeab530f (0x5583 bytes) 2024-09-22T01:14:17.372 INFO:teuthology.orchestra.run.smithi096.stdout:575 trunc from 0xeab5310 to 0xe5e131f 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:577 read 0x56b8cf2 thru 0x56bce6f (0x417e bytes) 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:578 read 0x954c4bd thru 0x955411b (0x7c5f bytes) 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:579 read 0x3f4b364 thru 0x3f550d1 (0x9d6e bytes) 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:580 read 0xbea2d64 thru 0xbeb0439 (0xd6d6 bytes) 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.394 INFO:teuthology.orchestra.run.smithi096.stdout:586 write 0xe04bb7c thru 0xe054bd8 (0x905d bytes) 2024-09-22T01:14:17.402 INFO:teuthology.orchestra.run.smithi096.stdout:588 write 0x7a2de88 thru 0x7a3a1e8 (0xc361 bytes) 2024-09-22T01:14:17.409 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:17.407+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:17.420 INFO:teuthology.orchestra.run.smithi096.stdout:589 write 0xe6a7cc8 thru 0xe6a9532 (0x186b bytes) 2024-09-22T01:14:17.443 INFO:teuthology.orchestra.run.smithi096.stdout:591 trunc from 0xe6a9533 to 0x3f623a8 2024-09-22T01:14:17.460 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:17.460 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:17.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:7677 3.12 e35/28 3:49695b41:::rbd_object_map.1124f75ab349:head v 35'49, mlcod=35'47)", 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:17.461334+0000", 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00061214899999999998, 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000653961, 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461334+0000", 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461334+0000", 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461336+0000", 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.719e-06 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:17.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461348+0000", 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1970000000000001e-05 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461378+0000", 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0164000000000001e-05 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461391+0000", 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2955999999999999e-05 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461464+0000", 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3052000000000005e-05 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:17.461485+0000", 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1027999999999999e-05 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:17.465 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-22T01:14:17.465 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:17.466 INFO:teuthology.orchestra.run.smithi096.stdout:592 read 0x7e4e9e thru 0x7e50bb (0x21e bytes) 2024-09-22T01:14:17.467 INFO:teuthology.orchestra.run.smithi096.stdout:594 punch from 0x33bc710 to 0x33c4cad, (0x859d bytes) 2024-09-22T01:14:17.467 INFO:teuthology.orchestra.run.smithi096.stdout:595 write 0x118a657 thru 0x1193b90 (0x953a bytes) 2024-09-22T01:14:17.476 INFO:teuthology.orchestra.run.smithi096.stdout:597 trunc from 0x3f623a8 to 0x67b3546 2024-09-22T01:14:17.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:17.488 INFO:teuthology.orchestra.run.smithi096.stdout:598 read 0x9f2d4d thru 0x9f759d (0x4851 bytes) 2024-09-22T01:14:17.490 INFO:teuthology.orchestra.run.smithi096.stdout:599 read 0xb424d9 thru 0xb5040b (0xdf33 bytes) 2024-09-22T01:14:17.497 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:17.498 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:17.510 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:17.507+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:17.515 INFO:teuthology.orchestra.run.smithi096.stdout:601 write 0xe1f35bd thru 0xe1fee80 (0xb8c4 bytes) 2024-09-22T01:14:17.517 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:17.517 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:17.541 INFO:teuthology.orchestra.run.smithi096.stdout:603 trunc from 0xe1fee81 to 0x19ebf10 2024-09-22T01:14:17.546 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:17.546 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:17.561 INFO:teuthology.orchestra.run.smithi096.stdout:604 writesame 0x59f711 thru 0x5a6982 data_size 0x39(0x7272 bytes) 2024-09-22T01:14:17.564 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:17.564 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:17.586 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:17.586 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:17.599 INFO:teuthology.orchestra.run.smithi096.stdout:605 writesame 0xb23cf1b thru 0xb240fe8 data_size 0x7(0x40ce bytes) 2024-09-22T01:14:17.599 INFO:teuthology.orchestra.run.smithi096.stdout:606 write 0x76e9cd9 thru 0x76ea28f (0x5b7 bytes) 2024-09-22T01:14:17.604 INFO:teuthology.orchestra.run.smithi096.stdout:608 punch from 0x263a78b to 0x26484f6, (0xdd6b bytes) 2024-09-22T01:14:17.610 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:17.611 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:17.611 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:17.611+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:17.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:14:17.618 INFO:teuthology.orchestra.run.smithi096.stdout:609 trunc from 0xb240fe9 to 0x2550b80 2024-09-22T01:14:17.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:17.619 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:17.640 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:611 read 0x1c0e7f3 thru 0x1c1dac8 (0xf2d6 bytes) 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:612 punch from 0x1288f4 to 0x135a8f, (0xd19b bytes) 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:614 read 0x1477c12 thru 0x147eaa0 (0x6e8f bytes) 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:615 read 0x1f126fb thru 0x1f16f03 (0x4809 bytes) 2024-09-22T01:14:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.645 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:17.645 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:17.645 INFO:teuthology.orchestra.run.smithi096.stdout:618 trunc from 0x2550b80 to 0x405ebe9 2024-09-22T01:14:17.658 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.658 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.662 INFO:teuthology.orchestra.run.smithi096.stdout:621 trunc from 0x405ebe9 to 0x154792f 2024-09-22T01:14:17.668 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:17.668 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:17.704 INFO:teuthology.orchestra.run.smithi096.stdout:623 write 0xdd2f7f9 thru 0xdd38292 (0x8a9a bytes) 2024-09-22T01:14:17.712 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:17.711+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:17.712 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.712 INFO:teuthology.orchestra.run.smithi096.stdout:625 write 0x3e8e8ff thru 0x3e8f59c (0xc9e bytes) 2024-09-22T01:14:17.718 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.718 INFO:teuthology.orchestra.run.smithi096.stdout:627 read 0x794a221 thru 0x79572a7 (0xd087 bytes) 2024-09-22T01:14:17.718 INFO:teuthology.orchestra.run.smithi096.stdout:628 write 0xa9bfcc1 thru 0xa9c54c5 (0x5805 bytes) 2024-09-22T01:14:17.745 INFO:teuthology.orchestra.run.smithi096.stdout:629 trunc from 0xdd38293 to 0x2741ff4 2024-09-22T01:14:17.766 INFO:teuthology.orchestra.run.smithi096.stdout:632 flatten 2024-09-22T01:14:17.774 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:17.774 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:17.793 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:17.821 INFO:teuthology.orchestra.run.smithi096.stdout:634 writesame 0x8b2cda1 thru 0x8b36db9 data_size 0x7(0xa019 bytes) 2024-09-22T01:14:17.821 INFO:teuthology.orchestra.run.smithi096.stdout:635 write 0x51a16a0 thru 0x51a6c75 (0x55d6 bytes) 2024-09-22T01:14:17.821 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:17.819+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:17.825 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.825 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.825 INFO:teuthology.orchestra.run.smithi096.stdout:639 punch from 0x1273e56 to 0x1276b8b, (0x2d35 bytes) 2024-09-22T01:14:17.831 INFO:teuthology.orchestra.run.smithi096.stdout:640 trunc from 0x8b36dba to 0x4a53e84 2024-09-22T01:14:17.846 INFO:teuthology.orchestra.run.smithi096.stdout:641 write 0x77277f thru 0x77385d (0x10df bytes) 2024-09-22T01:14:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:649 trunc from 0x4a53e84 to 0x7ac09c6 2024-09-22T01:14:17.869 INFO:teuthology.orchestra.run.smithi096.stdout:650 trunc from 0x7ac09c6 to 0xbcf95de 2024-09-22T01:14:17.882 INFO:teuthology.orchestra.run.smithi096.stdout:651 read 0x75e8d87 thru 0x75e8e80 (0xfa bytes) 2024-09-22T01:14:17.896 INFO:teuthology.orchestra.run.smithi096.stdout:652 write 0xe737895 thru 0xe744776 (0xcee2 bytes) 2024-09-22T01:14:17.905 INFO:teuthology.orchestra.run.smithi096.stdout:657 write 0x14bd8c4 thru 0x14cc127 (0xe864 bytes) 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:658 punch from 0x7d8d22 to 0x7e0b6a, (0x7e48 bytes) 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:662 read 0x2f91a94 thru 0x2f9f373 (0xd8e0 bytes) 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:663 read 0x3a267eb thru 0x3a35fa9 (0xf7bf bytes) 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:664 read 0x7e311a6 thru 0x7e35081 (0x3edc bytes) 2024-09-22T01:14:17.914 INFO:teuthology.orchestra.run.smithi096.stdout:667 writesame 0xb96d6bf thru 0xb96f2fb data_size 0x1(0x1c3d bytes) 2024-09-22T01:14:17.915 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:17.913+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:17.926 INFO:teuthology.orchestra.run.smithi096.stdout:669 write 0x5aad0a thru 0x5b6286 (0xb57d bytes) 2024-09-22T01:14:17.929 INFO:teuthology.orchestra.run.smithi096.stdout:672 writesame 0x6b1a3a1 thru 0x6b25fde data_size 0x7a(0xbc3e bytes) 2024-09-22T01:14:17.936 INFO:teuthology.orchestra.run.smithi096.stdout:674 punch from 0x245b933 to 0x245e628, (0x2cf5 bytes) 2024-09-22T01:14:17.937 INFO:teuthology.orchestra.run.smithi096.stdout:675 punch from 0x772d543 to 0x773577b, (0x8238 bytes) 2024-09-22T01:14:17.937 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:17.937 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:17.947 INFO:teuthology.orchestra.run.smithi096.stdout:677 trunc from 0xe744777 to 0x720acb5 2024-09-22T01:14:17.951 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:17.971 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.972 INFO:teuthology.orchestra.run.smithi096.stdout:679 punch from 0x3237a29 to 0x32405dc, (0x8bb3 bytes) 2024-09-22T01:14:17.972 INFO:teuthology.orchestra.run.smithi096.stdout:680 punch from 0xb81aed to 0xb826cd, (0xbe0 bytes) 2024-09-22T01:14:17.972 INFO:teuthology.orchestra.run.smithi096.stdout:681 punch from 0x6e0e3c6 to 0x6e11572, (0x31ac bytes) 2024-09-22T01:14:17.986 INFO:teuthology.orchestra.run.smithi096.stdout:683 trunc from 0x720acb5 to 0xedd173b 2024-09-22T01:14:17.998 INFO:teuthology.orchestra.run.smithi096.stdout:684 punch from 0xc1b6bf8 to 0xc1c22fd, (0xb705 bytes) 2024-09-22T01:14:17.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:17.999 INFO:teuthology.orchestra.run.smithi096.stdout:687 write 0x454b91c thru 0x455b0c7 (0xf7ac bytes) 2024-09-22T01:14:18.006 INFO:teuthology.orchestra.run.smithi096.stdout:688 writesame 0xb5dee04 thru 0xb5ece66 data_size 0x1(0xe063 bytes) 2024-09-22T01:14:18.035 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:18.037+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:18.102 INFO:teuthology.orchestra.run.smithi096.stdout:691 punch from 0xc9b9a4a to 0xc9c2cf3, (0x92a9 bytes) 2024-09-22T01:14:18.102 INFO:teuthology.orchestra.run.smithi096.stdout:692 read 0x47dd91c thru 0x47e9df8 (0xc4dd bytes) 2024-09-22T01:14:18.103 INFO:teuthology.orchestra.run.smithi096.stdout:693 writesame 0xb89fd7a thru 0xb8a2e4b data_size 0x3(0x30d2 bytes) 2024-09-22T01:14:18.115 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.115 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.116 INFO:teuthology.orchestra.run.smithi096.stdout:694 write 0xc2f78e2 thru 0xc300c2f (0x934e bytes) 2024-09-22T01:14:18.116 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:18.117+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:18.124 INFO:teuthology.orchestra.run.smithi096.stdout:695 write 0x2c5a84c thru 0x2c5ca25 (0x21da bytes) 2024-09-22T01:14:18.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:18.132 INFO:teuthology.orchestra.run.smithi096.stdout:696 write 0x409e90 thru 0x414d02 (0xae73 bytes) 2024-09-22T01:14:18.139 INFO:teuthology.orchestra.run.smithi096.stdout:697 write 0x8b43db7 thru 0x8b504ce (0xc718 bytes) 2024-09-22T01:14:18.146 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.146 INFO:teuthology.orchestra.run.smithi096.stdout:700 writesame 0x3b0a06 thru 0x3b731f data_size 0xb(0x691a bytes) 2024-09-22T01:14:18.155 INFO:teuthology.orchestra.run.smithi096.stdout:702 read 0xab0fe74 thru 0xab1222f (0x23bc bytes) 2024-09-22T01:14:18.156 INFO:teuthology.orchestra.run.smithi096.stdout:703 read 0xd964125 thru 0xd9645e5 (0x4c1 bytes) 2024-09-22T01:14:18.156 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.156 INFO:teuthology.orchestra.run.smithi096.stdout:707 punch from 0xc83912 to 0xc8a887, (0x6f75 bytes) 2024-09-22T01:14:18.192 INFO:teuthology.orchestra.run.smithi096.stdout:708 trunc from 0xedd173b to 0x1118ef4 2024-09-22T01:14:18.219 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:18.221+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:709 writesame 0x5fb54c0 thru 0x5fbc2e6 data_size 0x1(0x6e27 bytes) 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:711 punch from 0x3200ba to 0x32c98b, (0xc8d1 bytes) 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:712 read 0x1c9f477 thru 0x1ca062a (0x11b4 bytes) 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:713 punch from 0x389f99 to 0x393d8d, (0x9df4 bytes) 2024-09-22T01:14:18.247 INFO:teuthology.orchestra.run.smithi096.stdout:714 writesame 0x3375d69 thru 0x3375f66 data_size 0xf(0x1fe bytes) 2024-09-22T01:14:18.254 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.273 INFO:teuthology.orchestra.run.smithi096.stdout:716 trunc from 0x5fbc2e7 to 0xd178ef8 2024-09-22T01:14:18.286 INFO:teuthology.orchestra.run.smithi096.stdout:717 write 0x5013e0d thru 0x501fcb5 (0xbea9 bytes) 2024-09-22T01:14:18.294 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.294 INFO:teuthology.orchestra.run.smithi096.stdout:721 read 0x5200f3 thru 0x520ccc (0xbda bytes) 2024-09-22T01:14:18.294 INFO:teuthology.orchestra.run.smithi096.stdout:722 write 0x255d911 thru 0x2562b83 (0x5273 bytes) 2024-09-22T01:14:18.302 INFO:teuthology.orchestra.run.smithi096.stdout:724 write 0x59cbad4 thru 0x59cc11a (0x647 bytes) 2024-09-22T01:14:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.303 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.309 INFO:teuthology.orchestra.run.smithi096.stdout:725 write 0xc93cd47 thru 0xc940043 (0x32fd bytes) 2024-09-22T01:14:18.319 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:18.319 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:18.317+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:18.330 INFO:teuthology.orchestra.run.smithi096.stdout:726 trunc from 0xd178ef8 to 0x865680c 2024-09-22T01:14:18.350 INFO:teuthology.orchestra.run.smithi096.stdout:727 read 0x5163385 thru 0x516ea41 (0xb6bd bytes) 2024-09-22T01:14:18.350 INFO:teuthology.orchestra.run.smithi096.stdout:728 read 0x7c2feab thru 0x7c383d2 (0x8528 bytes) 2024-09-22T01:14:18.351 INFO:teuthology.orchestra.run.smithi096.stdout:729 writesame 0x6dca007 thru 0x6dd0847 data_size 0x1(0x6841 bytes) 2024-09-22T01:14:18.394 INFO:teuthology.orchestra.run.smithi096.stdout:730 write 0x81602e8 thru 0x816cf80 (0xcc99 bytes) 2024-09-22T01:14:18.404 INFO:teuthology.orchestra.run.smithi096.stdout:735 write 0x4759090 thru 0x47621a3 (0x9114 bytes) 2024-09-22T01:14:18.413 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.413 INFO:teuthology.orchestra.run.smithi096.stdout:737 read 0x2a52add thru 0x2a614f5 (0xea19 bytes) 2024-09-22T01:14:18.413 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.416 INFO:teuthology.orchestra.run.smithi096.stdout:739 trunc from 0x865680c to 0x937283c 2024-09-22T01:14:18.419 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:18.419+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:18.427 INFO:teuthology.orchestra.run.smithi096.stdout:740 write 0x6663ab5 thru 0x6672025 (0xe571 bytes) 2024-09-22T01:14:18.435 INFO:teuthology.orchestra.run.smithi096.stdout:741 write 0x663f676 thru 0x664720b (0x7b96 bytes) 2024-09-22T01:14:18.465 INFO:teuthology.orchestra.run.smithi096.stdout:742 write 0xaf9a51a thru 0xafa9d84 (0xf86b bytes) 2024-09-22T01:14:18.474 INFO:teuthology.orchestra.run.smithi096.stdout:743 write 0x822db16 thru 0x823052b (0x2a16 bytes) 2024-09-22T01:14:18.481 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.481 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:18.499 INFO:teuthology.orchestra.run.smithi096.stdout:744 write 0xcbde447 thru 0xcbe5dde (0x7998 bytes) 2024-09-22T01:14:18.512 INFO:teuthology.orchestra.run.smithi096.stdout:745 read 0x8931cfd thru 0x8938724 (0x6a28 bytes) 2024-09-22T01:14:18.512 INFO:teuthology.orchestra.run.smithi096.stdout:746 write 0x92b88f9 thru 0x92c0088 (0x7790 bytes) 2024-09-22T01:14:18.519 INFO:teuthology.orchestra.run.smithi096.stdout:747 punch from 0x926b9c8 to 0x926f886, (0x3ebe bytes) 2024-09-22T01:14:18.520 INFO:teuthology.orchestra.run.smithi096.stdout:748 read 0x91bbd62 thru 0x91c32bc (0x755b bytes) 2024-09-22T01:14:18.520 INFO:teuthology.orchestra.run.smithi096.stdout:749 write 0x9b2bda thru 0x9c10cc (0xe4f3 bytes) 2024-09-22T01:14:18.520 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:18.519+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:18.527 INFO:teuthology.orchestra.run.smithi096.stdout:751 write 0x72372a2 thru 0x72392bb (0x201a bytes) 2024-09-22T01:14:18.532 INFO:teuthology.orchestra.run.smithi096.stdout:753 writesame 0x511aed7 thru 0x5124feb data_size 0x7(0xa115 bytes) 2024-09-22T01:14:18.547 INFO:teuthology.orchestra.run.smithi096.stdout:754 read 0xfcb73b thru 0xfd4c59 (0x951f bytes) 2024-09-22T01:14:18.547 INFO:teuthology.orchestra.run.smithi096.stdout:757 write 0x5bd6fb6 thru 0x5be57ce (0xe819 bytes) 2024-09-22T01:14:18.556 INFO:teuthology.orchestra.run.smithi096.stdout:758 read 0x316a2cd thru 0x316aadf (0x813 bytes) 2024-09-22T01:14:18.556 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.557 INFO:teuthology.orchestra.run.smithi096.stdout:761 read 0xaaebb39 thru 0xaaec9ea (0xeb2 bytes) 2024-09-22T01:14:18.557 INFO:teuthology.orchestra.run.smithi096.stdout:763 write 0xb8c3369 thru 0xb8ceeeb (0xbb83 bytes) 2024-09-22T01:14:18.563 INFO:teuthology.orchestra.run.smithi096.stdout:764 write 0xed4781 thru 0xed53b9 (0xc39 bytes) 2024-09-22T01:14:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:765 writesame 0x7c80d13 thru 0x7c8f9fd data_size 0x17(0xeceb bytes) 2024-09-22T01:14:18.610 INFO:teuthology.orchestra.run.smithi096.stdout:766 trunc from 0xcbe5ddf to 0x120d34e 2024-09-22T01:14:18.617 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.617 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.621 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:18.619+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:14:18.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:18.655 INFO:teuthology.orchestra.run.smithi096.stdout:767 read 0x104b609 thru 0x1055131 (0x9b29 bytes) 2024-09-22T01:14:18.655 INFO:teuthology.orchestra.run.smithi096.stdout:768 read 0x105d28c thru 0x10693ed (0xc162 bytes) 2024-09-22T01:14:18.655 INFO:teuthology.orchestra.run.smithi096.stdout:770 trunc from 0x120d34e to 0x1752d9d 2024-09-22T01:14:18.663 INFO:teuthology.orchestra.run.smithi096.stdout:773 read 0x9b9134 thru 0x9be89e (0x576b bytes) 2024-09-22T01:14:18.664 INFO:teuthology.orchestra.run.smithi096.stdout:774 read 0x81572b thru 0x819626 (0x3efc bytes) 2024-09-22T01:14:18.689 INFO:teuthology.orchestra.run.smithi096.stdout:775 write 0xa4ba06f thru 0xa4be549 (0x44db bytes) 2024-09-22T01:14:18.694 INFO:teuthology.orchestra.run.smithi096.stdout:776 write 0x5992b9b thru 0x599e583 (0xb9e9 bytes) 2024-09-22T01:14:18.713 INFO:teuthology.orchestra.run.smithi096.stdout:777 write 0xc4c61c5 thru 0xc4d21f4 (0xc030 bytes) 2024-09-22T01:14:18.721 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:18.721+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:18.722 INFO:teuthology.orchestra.run.smithi096.stdout:778 read 0xaa26fe4 thru 0xaa345ec (0xd609 bytes) 2024-09-22T01:14:18.722 INFO:teuthology.orchestra.run.smithi096.stdout:779 punch from 0xf26598 to 0xf35a4a, (0xf4b2 bytes) 2024-09-22T01:14:18.722 INFO:teuthology.orchestra.run.smithi096.stdout:780 read 0x712e69c thru 0x71343b4 (0x5d19 bytes) 2024-09-22T01:14:18.722 INFO:teuthology.orchestra.run.smithi096.stdout:781 write 0x89cf6bc thru 0x89d3425 (0x3d6a bytes) 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:782 punch from 0x5090c1e to 0x509d26a, (0xc64c bytes) 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:783 read 0x7d4da73 thru 0x7d4f357 (0x18e5 bytes) 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:790 read 0x3b95b0d thru 0x3b9b0e6 (0x55da bytes) 2024-09-22T01:14:18.728 INFO:teuthology.orchestra.run.smithi096.stdout:791 write 0xa77c063 thru 0xa781df6 (0x5d94 bytes) 2024-09-22T01:14:18.735 INFO:teuthology.orchestra.run.smithi096.stdout:792 write 0x390f1e3 thru 0x3917ad2 (0x88f0 bytes) 2024-09-22T01:14:18.743 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.743 INFO:teuthology.orchestra.run.smithi096.stdout:794 read 0x49cb45f thru 0x49d5bc7 (0xa769 bytes) 2024-09-22T01:14:18.743 INFO:teuthology.orchestra.run.smithi096.stdout:796 punch from 0x432bfbf to 0x43316a3, (0x56e4 bytes) 2024-09-22T01:14:18.743 INFO:teuthology.orchestra.run.smithi096.stdout:797 punch from 0x6a60b2d to 0x6a6cf44, (0xc417 bytes) 2024-09-22T01:14:18.760 INFO:teuthology.orchestra.run.smithi096.stdout:798 write 0xd74dae1 thru 0xd75d2f1 (0xf811 bytes) 2024-09-22T01:14:18.766 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.766 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.767 INFO:teuthology.orchestra.run.smithi096.stdout:799 write 0xd4e13f1 thru 0xd4e4a1a (0x362a bytes) 2024-09-22T01:14:18.775 INFO:teuthology.orchestra.run.smithi096.stdout:800 punch from 0x69f7491 to 0x69f94cb, (0x203a bytes) 2024-09-22T01:14:18.781 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:18.793 INFO:teuthology.orchestra.run.smithi096.stdout:801 trunc from 0xd75d2f2 to 0x6fac67a 2024-09-22T01:14:18.817 INFO:teuthology.orchestra.run.smithi096.stdout:802 read 0x45f7226 thru 0x45fdfee (0x6dc9 bytes) 2024-09-22T01:14:18.818 INFO:teuthology.orchestra.run.smithi096.stdout:803 read 0x26f2115 thru 0x26fb75a (0x9646 bytes) 2024-09-22T01:14:18.818 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.818 INFO:teuthology.orchestra.run.smithi096.stdout:806 read 0x53520b5 thru 0x5355335 (0x3281 bytes) 2024-09-22T01:14:18.822 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:18.823+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:18.830 INFO:teuthology.orchestra.run.smithi096.stdout:807 trunc from 0x6fac67a to 0x28c66cf 2024-09-22T01:14:18.848 INFO:teuthology.orchestra.run.smithi096.stdout:808 read 0x255f048 thru 0x256b5ef (0xc5a8 bytes) 2024-09-22T01:14:18.881 INFO:teuthology.orchestra.run.smithi096.stdout:809 write 0xa563298 thru 0xa56cfed (0x9d56 bytes) 2024-09-22T01:14:18.887 INFO:teuthology.orchestra.run.smithi096.stdout:810 read 0x811df78 thru 0x8124920 (0x69a9 bytes) 2024-09-22T01:14:18.909 INFO:teuthology.orchestra.run.smithi096.stdout:811 trunc from 0xa56cfee to 0x2096d9b 2024-09-22T01:14:18.923 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:18.921+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:18.933 INFO:teuthology.orchestra.run.smithi096.stdout:812 read 0x1771096 thru 0x1780fe8 (0xff53 bytes) 2024-09-22T01:14:18.934 INFO:teuthology.orchestra.run.smithi096.stdout:814 read 0xf5f845 thru 0xf62eeb (0x36a7 bytes) 2024-09-22T01:14:18.951 INFO:teuthology.orchestra.run.smithi096.stdout:815 write 0x31ddd4d thru 0x31e5d5b (0x800f bytes) 2024-09-22T01:14:18.954 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:18.955 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:18.960 INFO:teuthology.orchestra.run.smithi096.stdout:817 punch from 0x17aa647 to 0x17b7fd8, (0xd991 bytes) 2024-09-22T01:14:18.960 INFO:teuthology.orchestra.run.smithi096.stdout:818 punch from 0x9832d0 to 0x98fe43, (0xcb73 bytes) 2024-09-22T01:14:18.960 INFO:teuthology.orchestra.run.smithi096.stdout:819 read 0x260a9d8 thru 0x2615986 (0xafaf bytes) 2024-09-22T01:14:18.960 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:18.961 INFO:teuthology.orchestra.run.smithi096.stdout:821 read 0xe7dbc3 thru 0xe7dec7 (0x305 bytes) 2024-09-22T01:14:18.961 INFO:teuthology.orchestra.run.smithi096.stdout:822 write 0xa6f129 thru 0xa7741f (0x82f7 bytes) 2024-09-22T01:14:18.970 INFO:teuthology.orchestra.run.smithi096.stdout:823 write 0x1a3db8c thru 0x1a4d599 (0xfa0e bytes) 2024-09-22T01:14:18.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:19.014 INFO:teuthology.orchestra.run.smithi096.stdout:824 write 0xc1924da thru 0xc196e0e (0x4935 bytes) 2024-09-22T01:14:19.020 INFO:teuthology.orchestra.run.smithi096.stdout:827 write 0x7cf2675 thru 0x7cfd81f (0xb1ab bytes) 2024-09-22T01:14:19.027 INFO:teuthology.orchestra.run.smithi096.stdout:829 read 0x9e60dd thru 0x9efc06 (0x9b2a bytes) 2024-09-22T01:14:19.028 INFO:teuthology.orchestra.run.smithi096.stdout:830 write 0x4b8c4bb thru 0x4b99f16 (0xda5c bytes) 2024-09-22T01:14:19.033 INFO:teuthology.orchestra.run.smithi096.stdout:831 write 0x7cdc4b2 thru 0x7ce0580 (0x40cf bytes) 2024-09-22T01:14:19.039 INFO:teuthology.orchestra.run.smithi096.stdout:832 read 0xb9c2a3b thru 0xb9c35b5 (0xb7b bytes) 2024-09-22T01:14:19.039 INFO:teuthology.orchestra.run.smithi096.stdout:833 write 0xa5775f1 thru 0xa57ab9e (0x35ae bytes) 2024-09-22T01:14:19.046 INFO:teuthology.orchestra.run.smithi096.stdout:834 write 0x5dac3c9 thru 0x5dad070 (0xca8 bytes) 2024-09-22T01:14:19.052 INFO:teuthology.orchestra.run.smithi096.stdout:835 write 0x551ae22 thru 0x5520358 (0x5537 bytes) 2024-09-22T01:14:19.060 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:19.061+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:19.060 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.090 INFO:teuthology.orchestra.run.smithi096.stdout:837 trunc from 0xc196e0f to 0x372127 2024-09-22T01:14:19.125 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:19.123+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:19.141 INFO:teuthology.orchestra.run.smithi096.stdout:839 writesame 0xe2131f thru 0xe2d989 data_size 0x1(0xc66b bytes) 2024-09-22T01:14:19.142 INFO:teuthology.orchestra.run.smithi096.stdout:841 writesame 0x4d0c4d thru 0x4dfbe4 data_size 0x58(0xef98 bytes) 2024-09-22T01:14:19.144 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:19.145 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:19.150 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:19.179 INFO:teuthology.orchestra.run.smithi096.stdout:843 trunc from 0xe2d98a to 0xc2f9576 2024-09-22T01:14:19.190 INFO:teuthology.orchestra.run.smithi096.stdout:844 write 0x4966ced thru 0x49700ef (0x9403 bytes) 2024-09-22T01:14:19.196 INFO:teuthology.orchestra.run.smithi096.stdout:845 write 0x22f1452 thru 0x22ff4b8 (0xe067 bytes) 2024-09-22T01:14:19.204 INFO:teuthology.orchestra.run.smithi096.stdout:846 read 0x93fea58 thru 0x9402fa1 (0x454a bytes) 2024-09-22T01:14:19.204 INFO:teuthology.orchestra.run.smithi096.stdout:847 punch from 0x110362e to 0x1105aaa, (0x247c bytes) 2024-09-22T01:14:19.204 INFO:teuthology.orchestra.run.smithi096.stdout:848 read 0x32fb6e0 thru 0x32fc9b6 (0x12d7 bytes) 2024-09-22T01:14:19.204 INFO:teuthology.orchestra.run.smithi096.stdout:849 read 0xa1e409d thru 0xa1ee078 (0x9fdc bytes) 2024-09-22T01:14:19.204 INFO:teuthology.orchestra.run.smithi096.stdout:851 read 0x352a1e6 thru 0x352bda7 (0x1bc2 bytes) 2024-09-22T01:14:19.218 INFO:teuthology.orchestra.run.smithi096.stdout:852 writesame 0xc306bf0 thru 0xc30aef0 data_size 0x1(0x4301 bytes) 2024-09-22T01:14:19.218 INFO:teuthology.orchestra.run.smithi096.stdout:853 writesame 0x27990e2 thru 0x279979f data_size 0x1(0x6be bytes) 2024-09-22T01:14:19.227 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.227 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.227 INFO:teuthology.orchestra.run.smithi096.stdout:859 read 0x932779b thru 0x932779c (0x2 bytes) 2024-09-22T01:14:19.228 INFO:teuthology.orchestra.run.smithi096.stdout:860 punch from 0xa0932fa to 0xa0942bb, (0xfc1 bytes) 2024-09-22T01:14:19.239 INFO:teuthology.orchestra.run.smithi096.stdout:861 write 0xc760d15 thru 0xc76433c (0x3628 bytes) 2024-09-22T01:14:19.246 INFO:teuthology.orchestra.run.smithi096.stdout:862 read 0xc336f21 thru 0xc343fb0 (0xd090 bytes) 2024-09-22T01:14:19.246 INFO:teuthology.orchestra.run.smithi096.stdout:863 read 0x3440471 thru 0x344fed4 (0xfa64 bytes) 2024-09-22T01:14:19.246 INFO:teuthology.orchestra.run.smithi096.stdout:865 writesame 0x7f3722b thru 0x7f38d85 data_size 0x1(0x1b5b bytes) 2024-09-22T01:14:19.247 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:19.249+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:19.262 INFO:teuthology.orchestra.run.smithi096.stdout:866 write 0xaefffb9 thru 0xaf0a8eb (0xa933 bytes) 2024-09-22T01:14:19.271 INFO:teuthology.orchestra.run.smithi096.stdout:867 write 0x3487955 thru 0x348c5ce (0x4c7a bytes) 2024-09-22T01:14:19.276 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.276 INFO:teuthology.orchestra.run.smithi096.stdout:870 write 0xa213bdd thru 0xa2182ab (0x46cf bytes) 2024-09-22T01:14:19.282 INFO:teuthology.orchestra.run.smithi096.stdout:871 punch from 0x7d3909c to 0x7d48666, (0xf5ca bytes) 2024-09-22T01:14:19.282 INFO:teuthology.orchestra.run.smithi096.stdout:872 read 0xb0ebc57 thru 0xb0efc02 (0x3fac bytes) 2024-09-22T01:14:19.282 INFO:teuthology.orchestra.run.smithi096.stdout:873 write 0x4b1fe52 thru 0x4b2bc03 (0xbdb2 bytes) 2024-09-22T01:14:19.294 INFO:teuthology.orchestra.run.smithi096.stdout:874 trunc from 0xc76433d to 0xe21fa75 2024-09-22T01:14:19.307 INFO:teuthology.orchestra.run.smithi096.stdout:875 write 0x1aff9b4 thru 0x1b0e3f5 (0xea42 bytes) 2024-09-22T01:14:19.317 INFO:teuthology.orchestra.run.smithi096.stdout:876 punch from 0xae9f967 to 0xaeaf3a9, (0xfa42 bytes) 2024-09-22T01:14:19.317 INFO:teuthology.orchestra.run.smithi096.stdout:877 punch from 0x82ca88c to 0x82d9a49, (0xf1bd bytes) 2024-09-22T01:14:19.317 INFO:teuthology.orchestra.run.smithi096.stdout:878 write 0x768260e thru 0x7682968 (0x35b bytes) 2024-09-22T01:14:19.324 INFO:teuthology.orchestra.run.smithi096.stdout:879 read 0x553845d thru 0x553a0ce (0x1c72 bytes) 2024-09-22T01:14:19.324 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.324 INFO:teuthology.orchestra.run.smithi096.stdout:882 write 0x8c8bd1e thru 0x8c9663f (0xa922 bytes) 2024-09-22T01:14:19.327 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:19.327+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:884 punch from 0x8570d7d to 0x857aef1, (0xa174 bytes) 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:885 read 0xd0d1d9 thru 0xd0eb01 (0x1929 bytes) 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:886 read 0x7f00b8b thru 0x7f0dca1 (0xd117 bytes) 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:887 read 0xcd12345 thru 0xcd162c0 (0x3f7c bytes) 2024-09-22T01:14:19.333 INFO:teuthology.orchestra.run.smithi096.stdout:888 read 0x9d5fd81 thru 0x9d6e811 (0xea91 bytes) 2024-09-22T01:14:19.334 INFO:teuthology.orchestra.run.smithi096.stdout:889 writesame 0x11abf2f thru 0x11b56f5 data_size 0x13(0x97c7 bytes) 2024-09-22T01:14:19.342 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:19.342 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:19.344 INFO:teuthology.orchestra.run.smithi096.stdout:891 writesame 0x939ab4e thru 0x93a1d37 data_size 0x2(0x71ea bytes) 2024-09-22T01:14:19.356 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:19.368 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.369 INFO:teuthology.orchestra.run.smithi096.stdout:894 writesame 0x8bff3ab thru 0x8c005e8 data_size 0x5(0x123e bytes) 2024-09-22T01:14:19.378 INFO:teuthology.orchestra.run.smithi096.stdout:895 read 0x59c7372 thru 0x59cd5c3 (0x6252 bytes) 2024-09-22T01:14:19.378 INFO:teuthology.orchestra.run.smithi096.stdout:896 read 0x5c9a00a thru 0x5c9d59d (0x3594 bytes) 2024-09-22T01:14:19.378 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.381 INFO:teuthology.orchestra.run.smithi096.stdout:900 trunc from 0xe21fa75 to 0xd576c4d 2024-09-22T01:14:19.397 INFO:teuthology.orchestra.run.smithi096.stdout:902 read 0x6bc3150 thru 0x6bd1c70 (0xeb21 bytes) 2024-09-22T01:14:19.398 INFO:teuthology.orchestra.run.smithi096.stdout:904 writesame 0x5f8af63 thru 0x5f99b85 data_size 0x1(0xec23 bytes) 2024-09-22T01:14:19.444 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:19.445+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:19.478 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.478 INFO:teuthology.orchestra.run.smithi096.stdout:907 writesame 0x9595a0 thru 0x959a00 data_size 0x1(0x461 bytes) 2024-09-22T01:14:19.492 INFO:teuthology.orchestra.run.smithi096.stdout:908 writesame 0x8bc44e3 thru 0x8bd3b2f data_size 0x1(0xf64d bytes) 2024-09-22T01:14:19.520 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:19.520 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:19.528 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:19.529+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:19.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:19.577 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:19.580 INFO:teuthology.orchestra.run.smithi096.stdout:910 trunc from 0xd576c4d to 0xeb1881f 2024-09-22T01:14:19.597 INFO:teuthology.orchestra.run.smithi096.stdout:911 writesame 0x95c0cc8 thru 0x95cb249 data_size 0x26(0xa582 bytes) 2024-09-22T01:14:19.607 INFO:teuthology.orchestra.run.smithi096.stdout:912 punch from 0x34dbaae to 0x34de4ce, (0x2a20 bytes) 2024-09-22T01:14:19.607 INFO:teuthology.orchestra.run.smithi096.stdout:913 writesame 0xe031b1a thru 0xe0359e2 data_size 0x1(0x3ec9 bytes) 2024-09-22T01:14:19.629 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:19.629+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:19.634 INFO:teuthology.orchestra.run.smithi096.stdout:915 read 0x84b53f7 thru 0x84b929d (0x3ea7 bytes) 2024-09-22T01:14:19.635 INFO:teuthology.orchestra.run.smithi096.stdout:916 write 0xc93b81c thru 0xc948576 (0xcd5b bytes) 2024-09-22T01:14:19.643 INFO:teuthology.orchestra.run.smithi096.stdout:920 punch from 0xc0b4f11 to 0xc0c037d, (0xb46c bytes) 2024-09-22T01:14:19.675 INFO:teuthology.orchestra.run.smithi096.stdout:921 trunc from 0xeb1881f to 0xcb256b 2024-09-22T01:14:19.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:19.709 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:19.709 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:19.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:19.752 INFO:teuthology.orchestra.run.smithi096.stdout:924 write 0x8bbc54c thru 0x8bc6925 (0xa3da bytes) 2024-09-22T01:14:19.760 INFO:teuthology.orchestra.run.smithi096.stdout:928 writesame 0x5480fe9 thru 0x548febb data_size 0x1(0xeed3 bytes) 2024-09-22T01:14:19.767 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:19.769+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:19.828 INFO:teuthology.orchestra.run.smithi096.stdout:929 trunc from 0x8bc6926 to 0x2156e1a 2024-09-22T01:14:19.830 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:19.831+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:19.853 INFO:teuthology.orchestra.run.smithi096.stdout:931 writesame 0x25f7e50 thru 0x25f9c41 data_size 0x1(0x1df2 bytes) 2024-09-22T01:14:19.855 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:19.855 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:19.863 INFO:teuthology.orchestra.run.smithi096.stdout:932 writesame 0x2c0fa40 thru 0x2c1618c data_size 0x7b(0x674d bytes) 2024-09-22T01:14:19.863 INFO:teuthology.orchestra.run.smithi096.stdout:933 read 0x824625 thru 0x82cf7b (0x8957 bytes) 2024-09-22T01:14:19.863 INFO:teuthology.orchestra.run.smithi096.stdout:934 punch from 0x21e4029 to 0x21ead41, (0x6d18 bytes) 2024-09-22T01:14:19.864 INFO:teuthology.orchestra.run.smithi096.stdout:935 trunc from 0x2c1618d to 0x3348911 2024-09-22T01:14:19.872 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:19.872 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:19.874 INFO:teuthology.orchestra.run.smithi096.stdout:936 read 0x2f08eb2 thru 0x2f16e42 (0xdf91 bytes) 2024-09-22T01:14:19.874 INFO:teuthology.orchestra.run.smithi096.stdout:937 read 0x2424c95 thru 0x242e4a5 (0x9811 bytes) 2024-09-22T01:14:19.889 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:19.890 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:19.893 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:19.893 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:19.897 INFO:teuthology.orchestra.run.smithi096.stdout:938 write 0xd49eb5b thru 0xd4a11a3 (0x2649 bytes) 2024-09-22T01:14:19.903 INFO:teuthology.orchestra.run.smithi096.stdout:939 read 0xa087eb3 thru 0xa08b3ad (0x34fb bytes) 2024-09-22T01:14:19.903 INFO:teuthology.orchestra.run.smithi096.stdout:941 write 0x482071c thru 0x4825ad6 (0x53bb bytes) 2024-09-22T01:14:19.907 INFO:teuthology.orchestra.run.smithi096.stdout:943 read 0x935517f thru 0x93567e0 (0x1662 bytes) 2024-09-22T01:14:19.907 INFO:teuthology.orchestra.run.smithi096.stdout:944 write 0x9185f66 thru 0x91936a0 (0xd73b bytes) 2024-09-22T01:14:19.907 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:19.908 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:19.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:19.911 INFO:teuthology.orchestra.run.smithi096.stdout:945 punch from 0x2c0d38f to 0x2c12dac, (0x5a1d bytes) 2024-09-22T01:14:19.911 INFO:teuthology.orchestra.run.smithi096.stdout:946 read 0xafbd567 thru 0xafbfcbd (0x2757 bytes) 2024-09-22T01:14:19.912 INFO:teuthology.orchestra.run.smithi096.stdout:947 writesame 0x9ef7a82 thru 0x9eff505 data_size 0x2(0x7a84 bytes) 2024-09-22T01:14:19.930 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:19.930 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:19.931 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:19.931+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:19.940 INFO:teuthology.orchestra.run.smithi096.stdout:950 trunc from 0xd4a11a4 to 0x78c5c1 2024-09-22T01:14:19.951 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:19.951 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:19.969 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:19.969 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:19.986 INFO:teuthology.orchestra.run.smithi096.stdout:951 write 0xd89d92e thru 0xd8a4416 (0x6ae9 bytes) 2024-09-22T01:14:19.991 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:19.992 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:20.007 INFO:teuthology.orchestra.run.smithi096.stdout:952 trunc from 0xd8a4417 to 0x59246a0 2024-09-22T01:14:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:953 punch from 0x10516c2 to 0x1054eb2, (0x37f0 bytes) 2024-09-22T01:14:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:957 punch from 0x346f716 to 0x347819e, (0x8a88 bytes) 2024-09-22T01:14:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:958 write 0x1d8fec8 thru 0x1d9afb8 (0xb0f1 bytes) 2024-09-22T01:14:20.032 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:20.031+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:20.044 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:20.044 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:20.044 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:20.047 INFO:teuthology.orchestra.run.smithi096.stdout:959 writesame 0xaa0cf92 thru 0xaa15697 data_size 0x15(0x8706 bytes) 2024-09-22T01:14:20.047 INFO:teuthology.orchestra.run.smithi096.stdout:960 read 0x3b35229 thru 0x3b3aeeb (0x5cc3 bytes) 2024-09-22T01:14:20.047 INFO:teuthology.orchestra.run.smithi096.stdout:961 write 0x6112282 thru 0x6119470 (0x71ef bytes) 2024-09-22T01:14:20.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:20.065 INFO:teuthology.orchestra.run.smithi096.stdout:962 write 0xc1d0f21 thru 0xc1d886b (0x794b bytes) 2024-09-22T01:14:20.070 INFO:teuthology.orchestra.run.smithi096.stdout:963 writesame 0x8d19e82 thru 0x8d25c72 data_size 0x19(0xbdf1 bytes) 2024-09-22T01:14:20.078 INFO:teuthology.orchestra.run.smithi096.stdout:965 write 0xa15878d thru 0xa15d48e (0x4d02 bytes) 2024-09-22T01:14:20.082 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.083 INFO:teuthology.orchestra.run.smithi096.stdout:967 read 0x6d5c43d thru 0x6d68463 (0xc027 bytes) 2024-09-22T01:14:20.083 INFO:teuthology.orchestra.run.smithi096.stdout:970 punch from 0x38fe469 to 0x3904097, (0x5c2e bytes) 2024-09-22T01:14:20.083 INFO:teuthology.orchestra.run.smithi096.stdout:974 read 0x5314bfa thru 0x531b1a2 (0x65a9 bytes) 2024-09-22T01:14:20.083 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.083 INFO:teuthology.orchestra.run.smithi096.stdout:980 read 0xba60ce4 thru 0xba6f42d (0xe74a bytes) 2024-09-22T01:14:20.099 INFO:teuthology.orchestra.run.smithi096.stdout:981 trunc from 0xc1d886c to 0xacb647 2024-09-22T01:14:20.120 INFO:teuthology.orchestra.run.smithi096.stdout:982 read 0x157bba thru 0x162d49 (0xb190 bytes) 2024-09-22T01:14:20.120 INFO:teuthology.orchestra.run.smithi096.stdout:984 read 0x18c373 thru 0x18ef32 (0x2bc0 bytes) 2024-09-22T01:14:20.120 INFO:teuthology.orchestra.run.smithi096.stdout:985 read 0x336c21 thru 0x338f7c (0x235c bytes) 2024-09-22T01:14:20.132 INFO:teuthology.orchestra.run.smithi096.stdout:988 trunc from 0xacb647 to 0x8fcb2c4 2024-09-22T01:14:20.133 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:20.133+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:991 punch from 0x872b9f3 to 0x8738194, (0xc7a1 bytes) 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:993 read 0x145a2c2 thru 0x1461e53 (0x7b92 bytes) 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:995 read 0x7e4537 thru 0x7f3d48 (0xf812 bytes) 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:996 punch from 0x795f16 to 0x7a0e99, (0xaf83 bytes) 2024-09-22T01:14:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:997 clone 7 order 22 su 1048576 sc 6 2024-09-22T01:14:20.187 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:20.188 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:20.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:20.234 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:20.233+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:20.335 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:20.337+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6542 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:20.376 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4255001409999997, 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012701107999999999, 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6542 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948069+0000", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948075+0000", 2024-09-22T01:14:20.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000003e-06 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948086+0000", 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0611000000000001e-05 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948098+0000", 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2251e-05 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955277+0000", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071787439999999999 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955323+0000", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955513+0000", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019068700000000001 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959026+0000", 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035129390000000001 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960685+0000", 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016590070000000001 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960726+0000", 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0630999999999998e-05 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960770+0000", 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3609000000000002e-05 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6543 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.4254404279999999, 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014741681, 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6543 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948129+0000", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948133+0000", 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980000000000003e-06 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948148+0000", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4711e-05 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948154+0000", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3849999999999999e-06 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955772+0000", 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076182760000000002 2024-09-22T01:14:20.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955821+0000", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8987999999999999e-05 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.956008+0000", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018672799999999999 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959043+0000", 2024-09-22T01:14:20.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030352629999999999 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962789+0000", 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037458420000000001 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962826+0000", 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6545999999999999e-05 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962870+0000", 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4118e-05 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7212 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 589824~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0524477829999999, 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012228655999999999, 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7212 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.84e-07 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321123+0000", 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321130+0000", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321135+0000", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6909999999999996e-06 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333201+0000", 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012066255999999999 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333274+0000", 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3170999999999995e-05 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333349+0000", 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5516e-05 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7213 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 720896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0524247530000004, 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012341655999999999, 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7213 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321147+0000", 2024-09-22T01:14:20.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321153+0000", 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321158+0000", 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0540000000000002e-06 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333385+0000", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012227098 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333432+0000", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.655e-05 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333486+0000", 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3984999999999999e-05 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7214 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 851968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0523707809999996, 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446456, 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7214 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321201+0000", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.576e-06 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321210+0000", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7979999999999995e-06 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321215+0000", 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333538+0000", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012322575000000001 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333594+0000", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6489000000000001e-05 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333644+0000", 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0234e-05 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7215 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0523458059999999, 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012531801, 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7215 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321239+0000", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5532999999999998e-05 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321246+0000", 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321255+0000", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4180000000000003e-06 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333660+0000", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012404851 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333711+0000", 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1298999999999998e-05 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333755+0000", 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7216 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0523049560000004, 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670147999999999, 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7216 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:20.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321266+0000", 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321273+0000", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321277+0000", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7060000000000003e-06 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333792+0000", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012514208000000001 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333862+0000", 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0078000000000002e-05 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333934+0000", 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2171999999999999e-05 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7217 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1245184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0522850979999996, 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012826168000000001, 2024-09-22T01:14:20.409 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7217 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321286+0000", 2024-09-22T01:14:20.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.764e-06 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321292+0000", 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.117e-06 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321297+0000", 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7269999999999998e-06 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333973+0000", 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012676481 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334042+0000", 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8674e-05 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334110+0000", 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7958000000000002e-05 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7218 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0522566969999998, 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012956225, 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7218 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321313+0000", 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321315+0000", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-06 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321321+0000", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321326+0000", 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.797e-06 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334165+0000", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01283831 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334209+0000", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4693000000000001e-05 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334268+0000", 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7219 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0522350170000001, 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013092405999999999, 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7219 2024-09-22T01:14:20.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:20.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321336+0000", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8840000000000001e-06 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321342+0000", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321346+0000", 2024-09-22T01:14:20.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334311+0000", 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012964361000000001 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334373+0000", 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1791000000000005e-05 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334426+0000", 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3365000000000002e-05 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7220 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1638400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0522161800000003, 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013199765, 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7220 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321353+0000", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.22e-07 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321355+0000", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321362+0000", 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-09-22T01:14:20.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321376+0000", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3372e-05 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334459+0000", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013083539999999999 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334508+0000", 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8754000000000001e-05 2024-09-22T01:14:20.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334552+0000", 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7377 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6296731749999998, 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013280372, 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7377 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743896+0000", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743899+0000", 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6869999999999999e-06 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743912+0000", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3144e-05 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743919+0000", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999994e-06 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747360+0000", 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034404280000000001 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747435+0000", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5312e-05 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747724+0000", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028894499999999998 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753696+0000", 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059718339999999996 2024-09-22T01:14:20.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757027+0000", 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033309030000000001 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757118+0000", 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757176+0000", 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7378 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.629642112, 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013472072999999999, 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7378 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743927+0000", 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743930+0000", 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8559999999999998e-06 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743935+0000", 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3199999999999999e-06 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743939+0000", 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3309999999999997e-06 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748128+0000", 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041883099999999998 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748217+0000", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9250000000000001e-05 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748581+0000", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036391 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754767+0000", 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061866630000000002 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757274+0000", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025063540000000001 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757344+0000", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0502999999999999e-05 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757399+0000", 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4242e-05 2024-09-22T01:14:20.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7379 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6296238990000003, 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013617644999999999, 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7379 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743948+0000", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743956+0000", 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-09-22T01:14:20.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743961+0000", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.498e-06 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749603+0000", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056418889999999998 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749676+0000", 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2832999999999999e-05 2024-09-22T01:14:20.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749988+0000", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031244399999999999 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756968+0000", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069804189999999999 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757467+0000", 2024-09-22T01:14:20.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049842199999999999 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757526+0000", 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9268000000000002e-05 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757562+0000", 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6173000000000001e-05 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7380 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6296021380000001, 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013877240000000001, 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7380 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743967+0000", 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743970+0000", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8439999999999998e-06 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743977+0000", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743981+0000", 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:14:20.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750397+0000", 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064165020000000001 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750522+0000", 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012448799999999999 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751087+0000", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000564947 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757205+0000", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061183619999999996 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757713+0000", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050744799999999997 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757790+0000", 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7334000000000002e-05 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757844+0000", 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3822000000000001e-05 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7381 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:20.449 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6295667040000001, 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014017416, 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7381 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:20.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744009+0000", 2024-09-22T01:14:20.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744018+0000", 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0340000000000002e-06 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744023+0000", 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751817+0000", 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077938979999999996 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751909+0000", 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2436999999999996e-05 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752214+0000", 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030542200000000002 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757581+0000", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053669049999999999 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757913+0000", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033169200000000002 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757965+0000", 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1456999999999999e-05 2024-09-22T01:14:20.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758019+0000", 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4775000000000003e-05 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7382 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.629538573, 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014167732000000001, 2024-09-22T01:14:20.455 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7382 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744033+0000", 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744048+0000", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744054+0000", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8159999999999999e-06 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752687+0000", 2024-09-22T01:14:20.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086338379999999996 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752828+0000", 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014017000000000001 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753320+0000", 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049253600000000002 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757591+0000", 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042712419999999997 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758086+0000", 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049453699999999997 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758149+0000", 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3261000000000003e-05 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758198+0000", 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7383 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.629508183, 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016002188000000001, 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7383 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744061+0000", 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744063+0000", 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744069+0000", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4829999999999999e-06 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744073+0000", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753747+0000", 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096735599999999995 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753844+0000", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7019000000000001e-05 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754147+0000", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030294600000000001 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758259+0000", 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004112826 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759947+0000", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001687287 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760009+0000", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2697e-05 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760063+0000", 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3202000000000003e-05 2024-09-22T01:14:20.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7384 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.6294894490000003, 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016118147999999999, 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7384 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744082+0000", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744087+0000", 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744091+0000", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754832+0000", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010740774999999999 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754970+0000", 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806199999999999 2024-09-22T01:14:20.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.755489+0000", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051922100000000005 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758312+0000", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028228089999999999 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760075+0000", 2024-09-22T01:14:20.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001762935 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760136+0000", 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1231999999999999e-05 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760197+0000", 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1033000000000003e-05 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7385 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.629472475, 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017662942000000001, 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7385 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.473 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:20.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744099+0000", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744113+0000", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3722e-05 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744120+0000", 2024-09-22T01:14:20.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756084+0000", 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011963061000000001 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756155+0000", 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1326000000000001e-05 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:20.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756461+0000", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030662199999999999 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759930+0000", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034685749999999998 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761621+0000", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001691173 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:20.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761703+0000", 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1729000000000003e-05 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761759+0000", 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6125999999999998e-05 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:20.478 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:20.480 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:20.480 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:20.479+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:20.581 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:20.579+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:20.652 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:20.653 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:20.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:14:20.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:20.711 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:20.711+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:20.782 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:20.779+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:20.833 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:20.834 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:20.834 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:20.849 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:20.889 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-22T01:14:20.889 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:20.887+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:20.895 INFO:teuthology.orchestra.run.smithi096.stdout:998 read 0x852a95a thru 0x852bfe0 (0x1687 bytes) 2024-09-22T01:14:20.895 INFO:teuthology.orchestra.run.smithi096.stdout:999 punch from 0x4e5483f to 0x4e5c5c6, (0x7d87 bytes) 2024-09-22T01:14:20.912 INFO:teuthology.orchestra.run.smithi096.stdout:1000 write 0xd2cf016 thru 0xd2d85f5 (0x95e0 bytes) 2024-09-22T01:14:20.919 INFO:teuthology.orchestra.run.smithi096.stdout:1001 write 0xc94c26d thru 0xc94fac8 (0x385c bytes) 2024-09-22T01:14:20.925 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.925 INFO:teuthology.orchestra.run.smithi096.stdout:1008 read 0x4e7668f thru 0x4e78318 (0x1c8a bytes) 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:1009 punch from 0x80c0375 to 0x80ce358, (0xdfe3 bytes) 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:1011 punch from 0x2dfb404 to 0x2e042e5, (0x8ee1 bytes) 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:1014 read 0xdae91c thru 0xdb2718 (0x3dfd bytes) 2024-09-22T01:14:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:1015 write 0xce63acb thru 0xce6ccab (0x91e1 bytes) 2024-09-22T01:14:20.945 INFO:teuthology.orchestra.run.smithi096.stdout:1016 trunc from 0xd2d85f6 to 0x4638a3f 2024-09-22T01:14:20.966 INFO:teuthology.orchestra.run.smithi096.stdout:1018 punch from 0x2b76a60 to 0x2b76d01, (0x2a1 bytes) 2024-09-22T01:14:20.966 INFO:teuthology.orchestra.run.smithi096.stdout:1019 punch from 0x30f7251 to 0x3106f58, (0xfd07 bytes) 2024-09-22T01:14:20.966 INFO:teuthology.orchestra.run.smithi096.stdout:1020 read 0x2f4771c thru 0x2f480bc (0x9a1 bytes) 2024-09-22T01:14:20.983 INFO:teuthology.orchestra.run.smithi096.stdout:1021 write 0xc8e206a thru 0xc8e613a (0x40d1 bytes) 2024-09-22T01:14:20.984 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:20.983+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:20.988 INFO:teuthology.orchestra.run.smithi096.stdout:1024 read 0x92bc546 thru 0x92bef65 (0x2a20 bytes) 2024-09-22T01:14:20.988 INFO:teuthology.orchestra.run.smithi096.stdout:1025 read 0xc3357ce thru 0xc343517 (0xdd4a bytes) 2024-09-22T01:14:20.988 INFO:teuthology.orchestra.run.smithi096.stdout:1026 read 0xbb2db49 thru 0xbb2e62c (0xae4 bytes) 2024-09-22T01:14:20.992 INFO:teuthology.orchestra.run.smithi096.stdout:1027 trunc from 0xc8e613b to 0x96d8973 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:1030 read 0x7797728 thru 0x77a72e7 (0xfbc0 bytes) 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:1032 punch from 0x10c6a82 to 0x10cbd80, (0x52fe bytes) 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:1034 punch from 0x68a2dd6 to 0x68b1630, (0xe85a bytes) 2024-09-22T01:14:21.006 INFO:teuthology.orchestra.run.smithi096.stdout:1036 read 0x20c72ab thru 0x20c7d1b (0xa71 bytes) 2024-09-22T01:14:21.009 INFO:teuthology.orchestra.run.smithi096.stdout:1037 trunc from 0x96d8973 to 0x790b54a 2024-09-22T01:14:21.015 INFO:teuthology.orchestra.run.smithi096.stdout:1039 punch from 0x357aff1 to 0x3586d05, (0xbd14 bytes) 2024-09-22T01:14:21.021 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 30.036301786999999, 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:21.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:21.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.786068829000001, 2024-09-22T01:14:21.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011381442, 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234702+0000", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234713+0000", 2024-09-22T01:14:21.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0507e-05 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234735+0000", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1800000000000001e-05 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234748+0000", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3583e-05 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243350+0000", 2024-09-22T01:14:21.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086011609999999995 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243483+0000", 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308499999999999 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243702+0000", 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021935499999999999 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245600+0000", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018977219999999999 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245927+0000", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032715799999999997 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246017+0000", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9752000000000004e-05 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246082+0000", 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5813000000000002e-05 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.785992542999999, 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013805090000000001, 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:21.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234779+0000", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-06 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234789+0000", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.042e-05 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234808+0000", 2024-09-22T01:14:21.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9256e-05 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234826+0000", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7825999999999998e-05 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244277+0000", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094506409999999992 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244411+0000", 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134513 2024-09-22T01:14:21.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244661+0000", 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024963399999999999 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245702+0000", 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001040736 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248290+0000", 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025886429999999998 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248450+0000", 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015967800000000001 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248582+0000", 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132316 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513928574000001, 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011449948999999999, 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506842+0000", 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506918+0000", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5715000000000003e-05 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506925+0000", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4599999999999997e-06 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506929+0000", 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2520000000000001e-06 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510441+0000", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035115350000000001 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510484+0000", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510560+0000", 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6184000000000001e-05 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513176+0000", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026151540000000002 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518234+0000", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050580019999999998 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518267+0000", 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3800999999999997e-05 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518291+0000", 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3855000000000001e-05 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513831572000001, 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011448767, 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506939+0000", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506942+0000", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8420000000000001e-06 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506945+0000", 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9349999999999996e-06 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506949+0000", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511010+0000", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040611680000000004 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511054+0000", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4255999999999998e-05 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511140+0000", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5841999999999996e-05 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513889+0000", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027490869999999999 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518324+0000", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044343489999999998 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518358+0000", 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4737999999999998e-05 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518387+0000", 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8572999999999999e-05 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513758503999998, 2024-09-22T01:14:21.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446914, 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:21.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507012+0000", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507015+0000", 2024-09-22T01:14:21.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507020+0000", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507023+0000", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511408+0000", 2024-09-22T01:14:21.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043849700000000002 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511451+0000", 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2735999999999999e-05 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511539+0000", 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7766e-05 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513894+0000", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023553100000000002 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519395+0000", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055011189999999996 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519431+0000", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5457999999999998e-05 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519458+0000", 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7682e-05 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513555726, 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015085121999999999, 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507215+0000", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507301+0000", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6791000000000003e-05 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507310+0000", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.653e-06 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507323+0000", 2024-09-22T01:14:21.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2648e-05 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511896+0000", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045732380000000003 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511954+0000", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8319000000000002e-05 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512028+0000", 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2024-09-22T01:14:21.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513897+0000", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001869587 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522235+0000", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083377590000000001 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522264+0000", 2024-09-22T01:14:21.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9431999999999999e-05 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522299+0000", 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4860999999999997e-05 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:21.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513391795, 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01505534, 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:21.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507379+0000", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507382+0000", 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.258e-06 2024-09-22T01:14:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507388+0000", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.263e-06 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507392+0000", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.771e-06 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512533+0000", 2024-09-22T01:14:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051413939999999997 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512618+0000", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4338000000000005e-05 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512742+0000", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012398199999999999 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513903+0000", 2024-09-22T01:14:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001161147 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522355+0000", 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084519360000000002 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522399+0000", 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4508000000000001e-05 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522433+0000", 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4177000000000002e-05 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.513309679999999, 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015058092, 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:21.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507461+0000", 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507463+0000", 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507475+0000", 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.146e-05 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507479+0000", 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1049999999999997e-06 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513231+0000", 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057521810000000003 2024-09-22T01:14:21.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513296+0000", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4764999999999994e-05 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513388+0000", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1847000000000003e-05 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.516279+0000", 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028914119999999999 2024-09-22T01:14:21.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522465+0000", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006186352 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522496+0000", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0988000000000002e-05 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522518+0000", 2024-09-22T01:14:21.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1792000000000001e-05 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.046088753999999, 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01429044, 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 131 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:21.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974682+0000", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974824+0000", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014190599999999999 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974839+0000", 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-09-22T01:14:21.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974849+0000", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8369999999999992e-06 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977235+0000", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023857219999999998 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977380+0000", 2024-09-22T01:14:21.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014566199999999999 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977640+0000", 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025971799999999998 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.980903+0000", 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032634040000000001 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988830+0000", 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260430000000007 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988904+0000", 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4016000000000004e-05 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988971+0000", 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7929000000000006e-05 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045900945, 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014424996000000001, 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.079 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974870+0000", 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:21.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974876+0000", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974889+0000", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2670000000000001e-05 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974896+0000", 2024-09-22T01:14:21.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9370000000000004e-06 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979031+0000", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041348749999999997 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979163+0000", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132142 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979423+0000", 2024-09-22T01:14:21.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025945100000000001 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981009+0000", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001585801 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989179+0000", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081698940000000005 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989243+0000", 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4480000000000006e-05 2024-09-22T01:14:21.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989294+0000", 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0742999999999999e-05 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045824926000002, 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484576000000001, 2024-09-22T01:14:21.084 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 133 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974946+0000", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0499999999999999e-06 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974952+0000", 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-22T01:14:21.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974964+0000", 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2164000000000001e-05 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974976+0000", 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1875999999999999e-05 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981087+0000", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061111660000000003 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981224+0000", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013626900000000001 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981437+0000", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021382899999999999 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985221+0000", 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037832999999999999 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989306+0000", 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004085506 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989379+0000", 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2909000000000003e-05 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989429+0000", 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0377000000000002e-05 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045721690000001, 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018165239999999999, 2024-09-22T01:14:21.090 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 134 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975049+0000", 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975179+0000", 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129722 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975197+0000", 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8113000000000001e-05 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975207+0000", 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5100000000000004e-06 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982219+0000", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070121760000000002 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982352+0000", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133249 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982617+0000", 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026466800000000002 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989070+0000", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064531299999999996 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.992931+0000", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003860774 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993092+0000", 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016092499999999999 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993213+0000", 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121836 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045545418, 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018294939, 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 135 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975225+0000", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975231+0000", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7699999999999998e-06 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975243+0000", 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1644000000000001e-05 2024-09-22T01:14:21.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975250+0000", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.503e-06 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983862+0000", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086120459999999999 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983989+0000", 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012705299999999999 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.984185+0000", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019606499999999999 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989159+0000", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049732120000000003 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993238+0000", 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040793449999999998 2024-09-22T01:14:21.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993372+0000", 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013405200000000001 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993519+0000", 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733899999999999 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045504184999999, 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018936457, 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 136 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8400000000000004e-07 2024-09-22T01:14:21.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975272+0000", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975284+0000", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2393e-05 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975292+0000", 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.763e-06 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985342+0000", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010050026 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985551+0000", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208755 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985988+0000", 2024-09-22T01:14:21.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043766699999999998 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989513+0000", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003525233 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994036+0000", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045223279999999999 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994117+0000", 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1549000000000001e-05 2024-09-22T01:14:21.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994202+0000", 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4735999999999996e-05 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045448349000001, 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019072927, 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:21.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975323+0000", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975483+0000", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160416 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975500+0000", 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7127000000000001e-05 2024-09-22T01:14:21.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975509+0000", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4399999999999994e-06 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987086+0000", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011576606999999999 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987414+0000", 2024-09-22T01:14:21.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032788700000000001 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987806+0000", 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039176100000000002 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993673+0000", 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058668970000000003 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994227+0000", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055439900000000002 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994330+0000", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010278 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994394+0000", 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4542999999999995e-05 2024-09-22T01:14:21.112 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045242866999999, 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019163698999999999, 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975528+0000", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.017e-06 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975534+0000", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975547+0000", 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2806000000000001e-05 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975554+0000", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989583+0000", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014028175 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989829+0000", 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024677100000000001 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.990235+0000", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040547400000000001 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994418+0000", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041833579999999999 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994534+0000", 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011541500000000001 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994623+0000", 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9703000000000005e-05 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994691+0000", 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7366000000000005e-05 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.118 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045203217000001, 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021194425999999999, 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:21.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975577+0000", 2024-09-22T01:14:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4669999999999993e-06 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975587+0000", 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975594+0000", 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1770000000000001e-06 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991392+0000", 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015798764 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991632+0000", 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023988199999999999 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991978+0000", 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034544099999999999 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996441+0000", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004463608 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996541+0000", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9623000000000005e-05 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996644+0000", 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103416 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996761+0000", 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116532 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045164427, 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022802374, 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975606+0000", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0299999999999998e-07 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975744+0000", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013788599999999999 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975767+0000", 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3003e-05 2024-09-22T01:14:21.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975781+0000", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.383e-05 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994785+0000", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019003995999999999 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994930+0000", 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144683 2024-09-22T01:14:21.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.995124+0000", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019457099999999999 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998089+0000", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029647440000000001 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998221+0000", 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178199999999999 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998301+0000", 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9839999999999995e-05 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998408+0000", 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107236 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.044973025000001, 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024814201000000001, 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975798+0000", 2024-09-22T01:14:21.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975807+0000", 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.533e-06 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975817+0000", 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9599999999999995e-06 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975824+0000", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3289999999999996e-06 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996882+0000", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021058038000000001 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997106+0000", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022396800000000001 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:21.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997464+0000", 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035745699999999999 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998789+0000", 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001325384 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000443+0000", 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001653493 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000521+0000", 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000611+0000", 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9662999999999999e-05 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.135 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:21.136 INFO:teuthology.orchestra.run.smithi096.stdout:1041 trunc from 0x790b54a to 0x5beb26 2024-09-22T01:14:21.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:1044 read 0xbf41e thru 0xca915 (0xb4f8 bytes) 2024-09-22T01:14:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:1045 write 0x7aad562 thru 0x7aadcf6 (0x795 bytes) 2024-09-22T01:14:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:1046 read 0x4fb2b53 thru 0x4fb7a17 (0x4ec5 bytes) 2024-09-22T01:14:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:1047 write 0x6df9a06 thru 0x6dfd0b4 (0x36af bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1048 punch from 0x5de596c to 0x5df2a78, (0xd10c bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1049 writesame 0x97fb036 thru 0x97fe994 data_size 0x1(0x395f bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1050 punch from 0x533056 to 0x53d3db, (0xa385 bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1051 read 0x139bd1c thru 0x13a095c (0x4c41 bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1052 read 0x1e06f7e thru 0x1e07824 (0x8a7 bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1053 write 0x2139a81 thru 0x213f436 (0x59b6 bytes) 2024-09-22T01:14:21.139 INFO:teuthology.orchestra.run.smithi096.stdout:1055 clone 8 order 19 su 8192 sc 14 2024-09-22T01:14:21.139 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:21.137+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:21.237 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:21.237+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:21.306 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:21.306 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:21.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:21.338 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:21.337+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:21.439 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:21.439+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:21.441 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:21.442 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:21.442 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:21.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:21.540 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:21.541+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1738 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 131072~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:21.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6181951859999995, 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014237117000000001, 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1738 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006357+0000", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5000999999999999e-05 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006366+0000", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006372+0000", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8379999999999998e-06 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013432+0000", 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070601079999999998 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013499+0000", 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7531000000000001e-05 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020579+0000", 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070795700000000003 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2025 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2916352~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6175043299999992, 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013872165000000001, 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2025 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:21.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007033+0000", 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007035+0000", 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9309999999999998e-06 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007042+0000", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8369999999999999e-06 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007046+0000", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6580000000000001e-06 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020772+0000", 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013725342 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020840+0000", 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8576000000000002e-05 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020905+0000", 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4469999999999998e-05 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2026 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6174838620000003, 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014009057, 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2026 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007056+0000", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007062+0000", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5570000000000001e-06 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007067+0000", 2024-09-22T01:14:21.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8029999999999996e-06 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020933+0000", 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013865475 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020999+0000", 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390999999999995e-05 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021062+0000", 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2027 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6174640670000002, 2024-09-22T01:14:21.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014158201, 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2027 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007074+0000", 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007083+0000", 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9889999999999998e-06 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007089+0000", 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5259999999999999e-06 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007094+0000", 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6530000000000002e-06 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021090+0000", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01399624 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021157+0000", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6538999999999994e-05 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021231+0000", 2024-09-22T01:14:21.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4640000000000004e-05 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2028 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3801088~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6174365510000008, 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014289873, 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2028 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007101+0000", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007103+0000", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007111+0000", 2024-09-22T01:14:21.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007122+0000", 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988000000000001e-05 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021258+0000", 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014136319 2024-09-22T01:14:21.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021326+0000", 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7642000000000001e-05 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021390+0000", 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4645000000000002e-05 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6174084769999997, 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:21.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:21.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6173878080000001, 2024-09-22T01:14:21.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:21.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:21.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6173574950000003, 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:21.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6173382410000006, 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:21.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.668 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6173201390000003, 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:21.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:21.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6172994159999998, 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:21.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6172789269999992, 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:21.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6172574849999997, 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:21.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:21.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.617226638, 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:21.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:21.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6171891120000002, 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6171569839999993, 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:21.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6171339769999999, 2024-09-22T01:14:21.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:21.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.5778201440000004, 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:21.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:21.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.5765329640000001, 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:21.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:21.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:21.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.5761887810000008, 2024-09-22T01:14:21.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:21.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:21.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:21.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:21.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:21.723 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:21.723 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:21.724 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:14:21.725 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:21.725+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:21.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:21.726 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:14:21.761 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone7 from 0x97fe995 (overlap 0x5beb26) to 0x3726b0 2024-09-22T01:14:21.825 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:21.825+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:21.863 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-22T01:14:21.904 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:21.905 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:21.924 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:21.926 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:21.925+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:22.023 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:22.059 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:22.060 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:22.060 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:22.060 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:22.059+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:22.077 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:22.162 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:22.163 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:22.163 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:22.163+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:22.180 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:22.180 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:22.196 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:22.196 INFO:teuthology.orchestra.run.smithi096.stdout:1058 punch from 0x93a9fcc to 0x93aaf22, (0xf56 bytes) 2024-09-22T01:14:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:22.197 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:22.214 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:22.215 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:22.224 INFO:teuthology.orchestra.run.smithi096.stdout:1059 write 0xded4580 thru 0xdee1872 (0xd2f3 bytes) 2024-09-22T01:14:22.229 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:22.229+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:22.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:22.240 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:22.241 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:22.250 INFO:teuthology.orchestra.run.smithi096.stdout:1062 punch from 0xc5c5518 to 0xc5cd7d2, (0x82ba bytes) 2024-09-22T01:14:22.250 INFO:teuthology.orchestra.run.smithi096.stdout:1063 punch from 0x191b99 to 0x1a0d82, (0xf1e9 bytes) 2024-09-22T01:14:22.250 INFO:teuthology.orchestra.run.smithi096.stdout:1064 write 0x2b02d68 thru 0x2b05491 (0x272a bytes) 2024-09-22T01:14:22.257 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:22.260 INFO:teuthology.orchestra.run.smithi096.stdout:1065 punch from 0x22165dd to 0x221b0e1, (0x4b04 bytes) 2024-09-22T01:14:22.260 INFO:teuthology.orchestra.run.smithi096.stdout:1066 read 0x5739583 thru 0x574390e (0xa38c bytes) 2024-09-22T01:14:22.261 INFO:teuthology.orchestra.run.smithi096.stdout:1067 read 0x36ea4fa thru 0x36ed87e (0x3385 bytes) 2024-09-22T01:14:22.261 INFO:teuthology.orchestra.run.smithi096.stdout:1069 punch from 0xc21c7e9 to 0xc21cae1, (0x2f8 bytes) 2024-09-22T01:14:22.261 INFO:teuthology.orchestra.run.smithi096.stdout:1070 write 0xc27ae14 thru 0xc27c1d7 (0x13c4 bytes) 2024-09-22T01:14:22.262 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:22.262 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:22.267 INFO:teuthology.orchestra.run.smithi096.stdout:1071 read 0xd17a87a thru 0xd182ce4 (0x846b bytes) 2024-09-22T01:14:22.267 INFO:teuthology.orchestra.run.smithi096.stdout:1072 read 0xb76393e thru 0xb76fe55 (0xc518 bytes) 2024-09-22T01:14:22.280 INFO:teuthology.orchestra.run.smithi096.stdout:1073 trunc from 0xdee1873 to 0x50320be 2024-09-22T01:14:22.283 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:22.283 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:22.306 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:22.306 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:22.325 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:22.325 INFO:teuthology.orchestra.run.smithi096.stdout:1077 write 0x2a61c7e thru 0x2a6810f (0x6492 bytes) 2024-09-22T01:14:22.330 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:22.327+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:22.346 INFO:teuthology.orchestra.run.smithi096.stdout:1078 trunc from 0x50320be to 0xc3b175a 2024-09-22T01:14:22.353 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:22.353 INFO:teuthology.orchestra.run.smithi096.stdout:1081 read 0x7612658 thru 0x761dde3 (0xb78c bytes) 2024-09-22T01:14:22.353 INFO:teuthology.orchestra.run.smithi096.stdout:1082 read 0xbd2295d thru 0xbd2ab64 (0x8208 bytes) 2024-09-22T01:14:22.353 INFO:teuthology.orchestra.run.smithi096.stdout:1083 write 0x7605431 thru 0x7608527 (0x30f7 bytes) 2024-09-22T01:14:22.364 INFO:teuthology.orchestra.run.smithi096.stdout:1085 read 0x6b7f01b thru 0x6b89e3f (0xae25 bytes) 2024-09-22T01:14:22.375 INFO:teuthology.orchestra.run.smithi096.stdout:1086 write 0xed5e6a6 thru 0xed66a6e (0x83c9 bytes) 2024-09-22T01:14:22.386 INFO:teuthology.orchestra.run.smithi096.stdout:1087 write 0x87e71ce thru 0x87f6a29 (0xf85c bytes) 2024-09-22T01:14:22.405 INFO:teuthology.orchestra.run.smithi096.stdout:1089 writesame 0xac21188 thru 0xac29c84 data_size 0x77(0x8afd bytes) 2024-09-22T01:14:22.410 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:22.410 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:22.414 INFO:teuthology.orchestra.run.smithi096.stdout:1092 writesame 0x52ced7e thru 0x52dd88c data_size 0x1d(0xeb0f bytes) 2024-09-22T01:14:22.421 INFO:teuthology.orchestra.run.smithi096.stdout:1094 write 0x62acca9 thru 0x62b416a (0x74c2 bytes) 2024-09-22T01:14:22.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:22.431 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:22.431+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:22.432 INFO:teuthology.orchestra.run.smithi096.stdout:1097 write 0xe75efb9 thru 0xe76a31a (0xb362 bytes) 2024-09-22T01:14:22.443 INFO:teuthology.orchestra.run.smithi096.stdout:1098 punch from 0x894606e to 0x8952ca8, (0xcc3a bytes) 2024-09-22T01:14:22.443 INFO:teuthology.orchestra.run.smithi096.stdout:1099 punch from 0x17f2d11 to 0x17f44fa, (0x17e9 bytes) 2024-09-22T01:14:22.443 INFO:teuthology.orchestra.run.smithi096.stdout:1100 write 0xe1f712e thru 0xe203b57 (0xca2a bytes) 2024-09-22T01:14:22.458 INFO:teuthology.orchestra.run.smithi096.stdout:1101 write 0x886ec0d thru 0x886f899 (0xc8d bytes) 2024-09-22T01:14:22.491 INFO:teuthology.orchestra.run.smithi096.stdout:1102 trunc from 0xed66a6f to 0x3ac7a37 2024-09-22T01:14:22.533 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:22.533+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:22.560 INFO:teuthology.orchestra.run.smithi096.stdout:1104 read 0x1995ff1 thru 0x19a354a (0xd55a bytes) 2024-09-22T01:14:22.560 INFO:teuthology.orchestra.run.smithi096.stdout:1106 read 0x1fa59e4 thru 0x1facfca (0x75e7 bytes) 2024-09-22T01:14:22.560 INFO:teuthology.orchestra.run.smithi096.stdout:1107 read 0x1d89525 thru 0x1d89bfa (0x6d6 bytes) 2024-09-22T01:14:22.561 INFO:teuthology.orchestra.run.smithi096.stdout:1108 punch from 0x120d66f to 0x120d6c1, (0x52 bytes) 2024-09-22T01:14:22.561 INFO:teuthology.orchestra.run.smithi096.stdout:1109 read 0x3173190 thru 0x31762f2 (0x3163 bytes) 2024-09-22T01:14:22.561 INFO:teuthology.orchestra.run.smithi096.stdout:1110 write 0x33f4e16 thru 0x33fe17f (0x936a bytes) 2024-09-22T01:14:22.566 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:22.566 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:22.575 INFO:teuthology.orchestra.run.smithi096.stdout:1111 writesame 0x2ed3105 thru 0x2ee2611 data_size 0xb(0xf50d bytes) 2024-09-22T01:14:22.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:22.596 INFO:teuthology.orchestra.run.smithi096.stdout:1112 punch from 0x160b8e to 0x162d9d, (0x220f bytes) 2024-09-22T01:14:22.601 INFO:teuthology.orchestra.run.smithi096.stdout:1113 trunc from 0x3ac7a37 to 0x21d1609 2024-09-22T01:14:22.633 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:22.631+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:22.667 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:14:22.691 INFO:teuthology.orchestra.run.smithi096.stdout:1114 read 0xa7b48a thru 0xa81fbf (0x6b36 bytes) 2024-09-22T01:14:22.692 INFO:teuthology.orchestra.run.smithi096.stdout:1115 punch from 0x1b3e958 to 0x1b459f8, (0x70a0 bytes) 2024-09-22T01:14:22.692 INFO:teuthology.orchestra.run.smithi096.stdout:1117 read 0x1e50a5b thru 0x1e57deb (0x7391 bytes) 2024-09-22T01:14:22.693 INFO:teuthology.orchestra.run.smithi096.stdout:1118 read 0x1f10950 thru 0x1f1443c (0x3aed bytes) 2024-09-22T01:14:22.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:14:22.708 INFO:teuthology.orchestra.run.smithi096.stdout:1119 writesame 0x6719957 thru 0x671f524 data_size 0x3(0x5bce bytes) 2024-09-22T01:14:22.709 INFO:teuthology.orchestra.run.smithi096.stdout:1120 punch from 0x2f59ecb to 0x2f634ce, (0x9603 bytes) 2024-09-22T01:14:22.723 INFO:teuthology.orchestra.run.smithi096.stdout:1121 write 0xbbf5572 thru 0xbbff1da (0x9c69 bytes) 2024-09-22T01:14:22.733 INFO:teuthology.orchestra.run.smithi096.stdout:1123 read 0x9a16d2 thru 0x9a1bb1 (0x4e0 bytes) 2024-09-22T01:14:22.733 INFO:teuthology.orchestra.run.smithi096.stdout:1124 read 0x938f71e thru 0x939aa72 (0xb355 bytes) 2024-09-22T01:14:22.734 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:22.731+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:22.734 INFO:teuthology.orchestra.run.smithi096.stdout:1125 read 0xb7fdad6 thru 0xb80aa87 (0xcfb2 bytes) 2024-09-22T01:14:22.735 INFO:teuthology.orchestra.run.smithi096.stdout:1126 read 0x9346b22 thru 0x9354c97 (0xe176 bytes) 2024-09-22T01:14:22.735 INFO:teuthology.orchestra.run.smithi096.stdout:1127 read 0x215526 thru 0x2228ea (0xd3c5 bytes) 2024-09-22T01:14:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:1129 read 0x110e961 thru 0x1110f05 (0x25a5 bytes) 2024-09-22T01:14:22.738 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:22.738 INFO:teuthology.orchestra.run.smithi096.stdout:1132 flatten 2024-09-22T01:14:22.770 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:22.770 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:22.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:22.835 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:22.835+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:22.935 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:22.937+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:22.940 INFO:teuthology.orchestra.run.smithi096.stdout:1137 writesame 0x62b756f thru 0x62c383c data_size 0x5(0xc2ce bytes) 2024-09-22T01:14:22.955 INFO:teuthology.orchestra.run.smithi096.stdout:1138 read 0x4553f78 thru 0x455f241 (0xb2ca bytes) 2024-09-22T01:14:22.955 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:22.955 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:1139 punch from 0x83d0ba5 to 0x83d7be4, (0x703f bytes) 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:1141 read 0xb47a437 thru 0xb4856a7 (0xb271 bytes) 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:1142 punch from 0x2d18b62 to 0x2d1e94e, (0x5dec bytes) 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:1143 punch from 0x8270be0 to 0x827ea63, (0xde83 bytes) 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:1144 punch from 0x9132248 to 0x913843d, (0x61f5 bytes) 2024-09-22T01:14:22.956 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:22.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:22.971 INFO:teuthology.orchestra.run.smithi096.stdout:1147 trunc from 0xbbff1db to 0x291532b 2024-09-22T01:14:22.988 INFO:teuthology.orchestra.run.smithi096.stdout:1148 punch from 0x17655b6 to 0x1771ec6, (0xc910 bytes) 2024-09-22T01:14:22.989 INFO:teuthology.orchestra.run.smithi096.stdout:1149 read 0x1557bfe thru 0x155f553 (0x7956 bytes) 2024-09-22T01:14:22.998 INFO:teuthology.orchestra.run.smithi096.stdout:1150 trunc from 0x291532b to 0x7eb5443 2024-09-22T01:14:23.006 INFO:teuthology.orchestra.run.smithi096.stdout:1151 write 0x587fb6f thru 0x588c0b9 (0xc54b bytes) 2024-09-22T01:14:23.020 INFO:teuthology.orchestra.run.smithi096.stdout:1153 write 0x3724f42 thru 0x3730f86 (0xc045 bytes) 2024-09-22T01:14:23.033 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.033 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.033 INFO:teuthology.orchestra.run.smithi096.stdout:1158 read 0x520b886 thru 0x5213134 (0x78af bytes) 2024-09-22T01:14:23.033 INFO:teuthology.orchestra.run.smithi096.stdout:1159 clone 9 order 20 su 32768 sc 2 2024-09-22T01:14:23.036 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:23.035+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:23.136 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:23.136 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:23.137 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:23.137+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:23.150 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:23.239 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:23.241+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:23.321 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:23.321 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:23.338 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:23.338 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:23.335+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:23.438 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:23.437+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:23.502 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:23.502 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:23.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:23.560 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:23.561+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:23.640 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:23.641+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:23.685 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:14:23.688 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:23.688 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:23.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:23.710 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:14:23.727 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:14:23.751 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:23.753+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:23.842 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:23.841+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:23.876 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:23.876 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:23.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:23.896 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-22T01:14:23.905 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.905 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.905 INFO:teuthology.orchestra.run.smithi096.stdout:1164 read 0x3741579 thru 0x3746be8 (0x5670 bytes) 2024-09-22T01:14:23.907 INFO:teuthology.orchestra.run.smithi096.stdout:1165 punch from 0xae1285 to 0xae98c1, (0x863c bytes) 2024-09-22T01:14:23.911 INFO:teuthology.orchestra.run.smithi096.stdout:1166 read 0x3582d01 thru 0x358a89a (0x7b9a bytes) 2024-09-22T01:14:23.913 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.913 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:23.913 INFO:teuthology.orchestra.run.smithi096.stdout:1169 punch from 0x99554b to 0x99aa22, (0x54d7 bytes) 2024-09-22T01:14:23.925 INFO:teuthology.orchestra.run.smithi096.stdout:1170 write 0xbc58c88 thru 0xbc5c297 (0x3610 bytes) 2024-09-22T01:14:23.930 INFO:teuthology.orchestra.run.smithi096.stdout:1171 writesame 0xa9b12fb thru 0xa9b46c8 data_size 0x13(0x33ce bytes) 2024-09-22T01:14:23.938 INFO:teuthology.orchestra.run.smithi096.stdout:1172 trunc from 0xbc5c298 to 0xe1662c7 2024-09-22T01:14:23.943 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:23.943+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:23.945 INFO:teuthology.orchestra.run.smithi096.stdout:1173 read 0x21ad763 thru 0x21b7f60 (0xa7fe bytes) 2024-09-22T01:14:23.959 INFO:teuthology.orchestra.run.smithi096.stdout:1176 trunc from 0xe1662c7 to 0x4d9dcec 2024-09-22T01:14:23.976 INFO:teuthology.orchestra.run.smithi096.stdout:1179 read 0x2c7f666 thru 0x2c89027 (0x99c2 bytes) 2024-09-22T01:14:23.983 INFO:teuthology.orchestra.run.smithi096.stdout:1180 trunc from 0x4d9dcec to 0x141ddc7 2024-09-22T01:14:23.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:24.020 INFO:teuthology.orchestra.run.smithi096.stdout:1182 write 0xa3f8de4 thru 0xa400ef9 (0x8116 bytes) 2024-09-22T01:14:24.026 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:24.027 INFO:teuthology.orchestra.run.smithi096.stdout:1184 read 0x48c721b thru 0x48cfd49 (0x8b2f bytes) 2024-09-22T01:14:24.027 INFO:teuthology.orchestra.run.smithi096.stdout:1185 writesame 0x8e2470 thru 0x8e9bac data_size 0x37(0x773d bytes) 2024-09-22T01:14:24.035 INFO:teuthology.orchestra.run.smithi096.stdout:1186 write 0x311250c thru 0x3120a5f (0xe554 bytes) 2024-09-22T01:14:24.044 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:24.043+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:24.050 INFO:teuthology.orchestra.run.smithi096.stdout:1188 write 0xa4dc366 thru 0xa4dc4e0 (0x17b bytes) 2024-09-22T01:14:24.055 INFO:teuthology.orchestra.run.smithi096.stdout:1189 writesame 0x7c3e6ee thru 0x7c49b94 data_size 0x1(0xb4a7 bytes) 2024-09-22T01:14:24.065 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:24.066 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:24.080 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:24.126 INFO:teuthology.orchestra.run.smithi096.stdout:1190 write 0xc2c8c23 thru 0xc2d33ab (0xa789 bytes) 2024-09-22T01:14:24.134 INFO:teuthology.orchestra.run.smithi096.stdout:1192 write 0x77bf200 thru 0x77c7de0 (0x8be1 bytes) 2024-09-22T01:14:24.140 INFO:teuthology.orchestra.run.smithi096.stdout:1193 write 0x389f38a thru 0x38a12fe (0x1f75 bytes) 2024-09-22T01:14:24.145 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:24.143+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:24.145 INFO:teuthology.orchestra.run.smithi096.stdout:1194 writesame 0xaf2e1df thru 0xaf2f2cb data_size 0x1(0x10ed bytes) 2024-09-22T01:14:24.155 INFO:teuthology.orchestra.run.smithi096.stdout:1196 clone 10 order 22 su 131072 sc 15 2024-09-22T01:14:24.200 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:24.200 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:24.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:24.246 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:24.243+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:24.325 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:24.335 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:24.335 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:24.347 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:24.345+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:24.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:24.448 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:24.447+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:24.479 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:24.493 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:24.493 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:24.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:24.495 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:24.507 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:24.513 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:24.513 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:24.529 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:24.530 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:24.548 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:24.549+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:24.550 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:24.550 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:24.570 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:24.570 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:24.591 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:24.591 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:24.611 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:24.611 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:24.637 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:24.637 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:24.637 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:24.649 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:24.649+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:24.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:24.686 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:14:24.689 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:14:24.719 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:14:24.750 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:24.749+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:24.784 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone9 intact 2024-09-22T01:14:24.833 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:24.833 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:24.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:24.833 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:24.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6542 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.8849321159999999, 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012701107999999999, 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6542 2024-09-22T01:14:24.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948069+0000", 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948075+0000", 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000003e-06 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948086+0000", 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0611000000000001e-05 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948098+0000", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2251e-05 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955277+0000", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071787439999999999 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955323+0000", 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955513+0000", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019068700000000001 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959026+0000", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035129390000000001 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960685+0000", 2024-09-22T01:14:24.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016590070000000001 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960726+0000", 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0630999999999998e-05 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960770+0000", 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3609000000000002e-05 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6543 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.8848724029999993, 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014741681, 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6543 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948129+0000", 2024-09-22T01:14:24.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948133+0000", 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980000000000003e-06 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948148+0000", 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4711e-05 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948154+0000", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3849999999999999e-06 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955772+0000", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076182760000000002 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955821+0000", 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8987999999999999e-05 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.956008+0000", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018672799999999999 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959043+0000", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030352629999999999 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962789+0000", 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037458420000000001 2024-09-22T01:14:24.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962826+0000", 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6545999999999999e-05 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962870+0000", 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4118e-05 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7212 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 589824~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5118797579999992, 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012228655999999999, 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7212 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321121+0000", 2024-09-22T01:14:24.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.84e-07 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321123+0000", 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.249e-06 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321130+0000", 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321135+0000", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6909999999999996e-06 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333201+0000", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012066255999999999 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333274+0000", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3170999999999995e-05 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333349+0000", 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5516e-05 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7213 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 720896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5118567279999997, 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012341655999999999, 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7213 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321144+0000", 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321147+0000", 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321153+0000", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2979999999999997e-06 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321158+0000", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0540000000000002e-06 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333385+0000", 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012227098 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333432+0000", 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.655e-05 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333486+0000", 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3984999999999999e-05 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.854 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7214 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 851968~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5118027559999998, 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446456, 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7214 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321198+0000", 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321201+0000", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.576e-06 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321210+0000", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7979999999999995e-06 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321215+0000", 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1599999999999997e-06 2024-09-22T01:14:24.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333538+0000", 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012322575000000001 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333594+0000", 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6489000000000001e-05 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333644+0000", 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0234e-05 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7215 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5117777809999993, 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012531801, 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7215 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:24.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321239+0000", 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5532999999999998e-05 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321246+0000", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321255+0000", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4180000000000003e-06 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333660+0000", 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012404851 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333711+0000", 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1298999999999998e-05 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333755+0000", 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7216 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5117369309999997, 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670147999999999, 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7216 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2024-09-22T01:14:24.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321266+0000", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321273+0000", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321277+0000", 2024-09-22T01:14:24.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7060000000000003e-06 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333792+0000", 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012514208000000001 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333862+0000", 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0078000000000002e-05 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333934+0000", 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2171999999999999e-05 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7217 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1245184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5117170729999998, 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012826168000000001, 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7217 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:24.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321286+0000", 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.764e-06 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321292+0000", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.117e-06 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321297+0000", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7269999999999998e-06 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333973+0000", 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012676481 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334042+0000", 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8674e-05 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334110+0000", 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7958000000000002e-05 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.872 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7218 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.511688672, 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012956225, 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7218 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321313+0000", 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:14:24.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321315+0000", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-06 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321321+0000", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321326+0000", 2024-09-22T01:14:24.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.797e-06 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334165+0000", 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01283831 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334209+0000", 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4693000000000001e-05 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334268+0000", 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7219 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5116669920000003, 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013092405999999999, 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7219 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321336+0000", 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8840000000000001e-06 2024-09-22T01:14:24.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321342+0000", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321346+0000", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334311+0000", 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012964361000000001 2024-09-22T01:14:24.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334373+0000", 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1791000000000005e-05 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334426+0000", 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3365000000000002e-05 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7220 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1638400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.5116481549999996, 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013199765, 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7220 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321353+0000", 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.22e-07 2024-09-22T01:14:24.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321355+0000", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321362+0000", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321376+0000", 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3372e-05 2024-09-22T01:14:24.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334459+0000", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013083539999999999 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334508+0000", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8754000000000001e-05 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334552+0000", 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2024-09-22T01:14:24.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7377 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.08910515, 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013280372, 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7377 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743896+0000", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743899+0000", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6869999999999999e-06 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743912+0000", 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3144e-05 2024-09-22T01:14:24.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743919+0000", 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999994e-06 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747360+0000", 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034404280000000001 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747435+0000", 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5312e-05 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747724+0000", 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028894499999999998 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753696+0000", 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059718339999999996 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757027+0000", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033309030000000001 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757118+0000", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757176+0000", 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2024-09-22T01:14:24.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7378 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0890740870000002, 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013472072999999999, 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.892 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7378 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743927+0000", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743930+0000", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8559999999999998e-06 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743935+0000", 2024-09-22T01:14:24.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3199999999999999e-06 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743939+0000", 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3309999999999997e-06 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748128+0000", 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041883099999999998 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748217+0000", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9250000000000001e-05 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748581+0000", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036391 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754767+0000", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061866630000000002 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757274+0000", 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025063540000000001 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757344+0000", 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0502999999999999e-05 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757399+0000", 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4242e-05 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7379 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0890558739999996, 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013617644999999999, 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7379 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.898 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743948+0000", 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743956+0000", 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743961+0000", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.498e-06 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749603+0000", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056418889999999998 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749676+0000", 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2832999999999999e-05 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749988+0000", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031244399999999999 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756968+0000", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069804189999999999 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757467+0000", 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049842199999999999 2024-09-22T01:14:24.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757526+0000", 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9268000000000002e-05 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757562+0000", 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6173000000000001e-05 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7380 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0890341130000003, 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013877240000000001, 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7380 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743967+0000", 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:14:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743970+0000", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8439999999999998e-06 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743977+0000", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743981+0000", 2024-09-22T01:14:24.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750397+0000", 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064165020000000001 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750522+0000", 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012448799999999999 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751087+0000", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000564947 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757205+0000", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061183619999999996 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757713+0000", 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050744799999999997 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757790+0000", 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7334000000000002e-05 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757844+0000", 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3822000000000001e-05 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7381 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0889986789999995, 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014017416, 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7381 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744009+0000", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744018+0000", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0340000000000002e-06 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744023+0000", 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-22T01:14:24.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751817+0000", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077938979999999996 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751909+0000", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2436999999999996e-05 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752214+0000", 2024-09-22T01:14:24.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030542200000000002 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757581+0000", 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053669049999999999 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757913+0000", 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033169200000000002 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757965+0000", 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1456999999999999e-05 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758019+0000", 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4775000000000003e-05 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7382 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:24.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0889705480000007, 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014167732000000001, 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7382 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:24.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744033+0000", 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744048+0000", 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744054+0000", 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8159999999999999e-06 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752687+0000", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086338379999999996 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752828+0000", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014017000000000001 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753320+0000", 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049253600000000002 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757591+0000", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042712419999999997 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758086+0000", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049453699999999997 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758149+0000", 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3261000000000003e-05 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758198+0000", 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7383 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0889401579999998, 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016002188000000001, 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7383 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744061+0000", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744063+0000", 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744069+0000", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4829999999999999e-06 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744073+0000", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753747+0000", 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096735599999999995 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753844+0000", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7019000000000001e-05 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754147+0000", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030294600000000001 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758259+0000", 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004112826 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759947+0000", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001687287 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760009+0000", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2697e-05 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760063+0000", 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3202000000000003e-05 2024-09-22T01:14:24.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7384 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0889214240000005, 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016118147999999999, 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7384 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744082+0000", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744087+0000", 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:14:24.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744091+0000", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754832+0000", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010740774999999999 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754970+0000", 2024-09-22T01:14:24.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806199999999999 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.755489+0000", 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051922100000000005 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758312+0000", 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028228089999999999 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760075+0000", 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001762935 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760136+0000", 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1231999999999999e-05 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760197+0000", 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1033000000000003e-05 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.932 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7385 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 8.0889044499999994, 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017662942000000001, 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:24.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7385 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744099+0000", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744113+0000", 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3722e-05 2024-09-22T01:14:24.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744120+0000", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756084+0000", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011963061000000001 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756155+0000", 2024-09-22T01:14:24.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1326000000000001e-05 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756461+0000", 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030662199999999999 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759930+0000", 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034685749999999998 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:24.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761621+0000", 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001691173 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761703+0000", 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1729000000000003e-05 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761759+0000", 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6125999999999998e-05 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:24.939 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:24.939 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:24.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:24.941 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:24.941+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:24.972 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-22T01:14:25.041 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:25.039+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:25.095 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:25.095 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:25.095 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:25.111 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:25.142 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:25.141+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:25.233 INFO:teuthology.orchestra.run.smithi096.stdout:1197 punch from 0x8d96a6e to 0x8da11ea, (0xa77c bytes) 2024-09-22T01:14:25.242 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:25.241+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:25.244 INFO:teuthology.orchestra.run.smithi096.stdout:1198 trunc from 0xc2d33ac to 0x86c8638 2024-09-22T01:14:25.279 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:25.279 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:25.279 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:25.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:25.299 INFO:teuthology.orchestra.run.smithi096.stdout:1199 write 0xe42ce79 thru 0xe4339a9 (0x6b31 bytes) 2024-09-22T01:14:25.304 INFO:teuthology.orchestra.run.smithi096.stdout:1200 read 0x84c4a92 thru 0x84cb58a (0x6af9 bytes) 2024-09-22T01:14:25.305 INFO:teuthology.orchestra.run.smithi096.stdout:1202 punch from 0xb97586f to 0xb975ab8, (0x249 bytes) 2024-09-22T01:14:25.305 INFO:teuthology.orchestra.run.smithi096.stdout:1203 read 0x62e9c98 thru 0x62f708d (0xd3f6 bytes) 2024-09-22T01:14:25.305 INFO:teuthology.orchestra.run.smithi096.stdout:1204 write 0xcc82e96 thru 0xcc8a2b4 (0x741f bytes) 2024-09-22T01:14:25.309 INFO:teuthology.orchestra.run.smithi096.stdout:1205 punch from 0x7b7ac09 to 0x7b7cd99, (0x2190 bytes) 2024-09-22T01:14:25.309 INFO:teuthology.orchestra.run.smithi096.stdout:1206 write 0xae032b0 thru 0xae0a966 (0x76b7 bytes) 2024-09-22T01:14:25.312 INFO:teuthology.orchestra.run.smithi096.stdout:1207 read 0x60f04de thru 0x60f2f2b (0x2a4e bytes) 2024-09-22T01:14:25.312 INFO:teuthology.orchestra.run.smithi096.stdout:1208 read 0x142d2d1 thru 0x143773f (0xa46f bytes) 2024-09-22T01:14:25.313 INFO:teuthology.orchestra.run.smithi096.stdout:1209 read 0x3e8a5ce thru 0x3e90dd6 (0x6809 bytes) 2024-09-22T01:14:25.313 INFO:teuthology.orchestra.run.smithi096.stdout:1213 write 0x8ccac35 thru 0x8cd9cdb (0xf0a7 bytes) 2024-09-22T01:14:25.317 INFO:teuthology.orchestra.run.smithi096.stdout:1219 writesame 0x511eaf5 thru 0x51286fd data_size 0x5(0x9c09 bytes) 2024-09-22T01:14:25.335 INFO:teuthology.orchestra.run.smithi096.stdout:1220 trunc from 0xe4339aa to 0x5c53752 2024-09-22T01:14:25.343 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:25.345+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:25.381 INFO:teuthology.orchestra.run.smithi096.stdout:1221 read 0x3039a9 thru 0x30896b (0x4fc3 bytes) 2024-09-22T01:14:25.382 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.382 INFO:teuthology.orchestra.run.smithi096.stdout:1224 read 0x2b153b1 thru 0x2b1fb0a (0xa75a bytes) 2024-09-22T01:14:25.382 INFO:teuthology.orchestra.run.smithi096.stdout:1225 write 0x534fe64 thru 0x535981c (0x99b9 bytes) 2024-09-22T01:14:25.386 INFO:teuthology.orchestra.run.smithi096.stdout:1226 read 0x124d795 thru 0x1254a19 (0x7285 bytes) 2024-09-22T01:14:25.395 INFO:teuthology.orchestra.run.smithi096.stdout:1227 write 0x62c977c thru 0x62d029d (0x6b22 bytes) 2024-09-22T01:14:25.399 INFO:teuthology.orchestra.run.smithi096.stdout:1228 read 0x961696 thru 0x970cc1 (0xf62c bytes) 2024-09-22T01:14:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:1230 punch from 0x23f897 to 0x24e6a7, (0xee10 bytes) 2024-09-22T01:14:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:1233 punch from 0x1c069c8 to 0x1c0cc05, (0x623d bytes) 2024-09-22T01:14:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:1234 write 0x2d07855 thru 0x2d0d32b (0x5ad7 bytes) 2024-09-22T01:14:25.409 INFO:teuthology.orchestra.run.smithi096.stdout:1235 write 0x1d339a0 thru 0x1d3ce79 (0x94da bytes) 2024-09-22T01:14:25.423 INFO:teuthology.orchestra.run.smithi096.stdout:1236 read 0x613b1a2 thru 0x613f2c3 (0x4122 bytes) 2024-09-22T01:14:25.424 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.424 INFO:teuthology.orchestra.run.smithi096.stdout:1238 read 0x1e7aa1f thru 0x1e85d2d (0xb30f bytes) 2024-09-22T01:14:25.429 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:25.429 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.444804697000002, 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.430 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:25.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:25.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:25.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.194571738999997, 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011381442, 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 36 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:25.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234701+0000", 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234702+0000", 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5060000000000001e-06 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234713+0000", 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0507e-05 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234735+0000", 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1800000000000001e-05 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234748+0000", 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3583e-05 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243350+0000", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086011609999999995 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243483+0000", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013308499999999999 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.243702+0000", 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021935499999999999 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245600+0000", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018977219999999999 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245927+0000", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032715799999999997 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246017+0000", 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9752000000000004e-05 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.246082+0000", 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5813000000000002e-05 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.194495453000002, 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013805090000000001, 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234777+0000", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234779+0000", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4270000000000001e-06 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234789+0000", 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.042e-05 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234808+0000", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9256e-05 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.234826+0000", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7825999999999998e-05 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244277+0000", 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094506409999999992 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244411+0000", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134513 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.244661+0000", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024963399999999999 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.245702+0000", 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001040736 2024-09-22T01:14:25.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248290+0000", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025886429999999998 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248450+0000", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015967800000000001 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.248582+0000", 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132316 2024-09-22T01:14:25.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.922431484000001, 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011449948999999999, 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506841+0000", 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506842+0000", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506918+0000", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5715000000000003e-05 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506925+0000", 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4599999999999997e-06 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506929+0000", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2520000000000001e-06 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510441+0000", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035115350000000001 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510484+0000", 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3321000000000001e-05 2024-09-22T01:14:25.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.510560+0000", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6184000000000001e-05 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513176+0000", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026151540000000002 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518234+0000", 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050580019999999998 2024-09-22T01:14:25.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518267+0000", 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3800999999999997e-05 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518291+0000", 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3855000000000001e-05 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.922334481999997, 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011448767, 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506938+0000", 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506939+0000", 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506942+0000", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8420000000000001e-06 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506945+0000", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9349999999999996e-06 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.506949+0000", 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511010+0000", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040611680000000004 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511054+0000", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4255999999999998e-05 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511140+0000", 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5841999999999996e-05 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513889+0000", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027490869999999999 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518324+0000", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044343489999999998 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518358+0000", 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4737999999999998e-05 2024-09-22T01:14:25.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.518387+0000", 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8572999999999999e-05 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.922261413999998, 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012446914, 2024-09-22T01:14:25.459 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507011+0000", 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507012+0000", 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8299999999999997e-07 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507015+0000", 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8100000000000002e-06 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507020+0000", 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507023+0000", 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.241e-06 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511408+0000", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043849700000000002 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511451+0000", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2735999999999999e-05 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511539+0000", 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7766e-05 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513894+0000", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023553100000000002 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519395+0000", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055011189999999996 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519431+0000", 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5457999999999998e-05 2024-09-22T01:14:25.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.519458+0000", 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7682e-05 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.922058636000003, 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015085121999999999, 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507214+0000", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507215+0000", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507301+0000", 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6791000000000003e-05 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507310+0000", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.653e-06 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507323+0000", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2648e-05 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511896+0000", 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045732380000000003 2024-09-22T01:14:25.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.511954+0000", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8319000000000002e-05 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512028+0000", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3308999999999999e-05 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513897+0000", 2024-09-22T01:14:25.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001869587 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522235+0000", 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083377590000000001 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522264+0000", 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9431999999999999e-05 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522299+0000", 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4860999999999997e-05 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.921894705, 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01505534, 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507378+0000", 2024-09-22T01:14:25.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507379+0000", 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507382+0000", 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.258e-06 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507388+0000", 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.263e-06 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507392+0000", 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.771e-06 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512533+0000", 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0051413939999999997 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512618+0000", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4338000000000005e-05 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.512742+0000", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012398199999999999 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513903+0000", 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001161147 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522355+0000", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084519360000000002 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522399+0000", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4508000000000001e-05 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522433+0000", 2024-09-22T01:14:25.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4177000000000002e-05 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.921812590000002, 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015058092, 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507460+0000", 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507461+0000", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000002e-07 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507463+0000", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7310000000000002e-06 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507475+0000", 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.146e-05 2024-09-22T01:14:25.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.507479+0000", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1049999999999997e-06 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513231+0000", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057521810000000003 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513296+0000", 2024-09-22T01:14:25.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4764999999999994e-05 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.513388+0000", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1847000000000003e-05 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.516279+0000", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028914119999999999 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522465+0000", 2024-09-22T01:14:25.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006186352 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522496+0000", 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0988000000000002e-05 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.522518+0000", 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1792000000000001e-05 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:131 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454591663999999, 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01429044, 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 131 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.483 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974681+0000", 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974682+0000", 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974824+0000", 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014190599999999999 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974839+0000", 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5096000000000001e-05 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974849+0000", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8369999999999992e-06 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977235+0000", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023857219999999998 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977380+0000", 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014566199999999999 2024-09-22T01:14:25.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.977640+0000", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025971799999999998 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.980903+0000", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032634040000000001 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988830+0000", 2024-09-22T01:14:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260430000000007 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988904+0000", 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4016000000000004e-05 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.988971+0000", 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7929000000000006e-05 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:132 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454403855000002, 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014424996000000001, 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 132 2024-09-22T01:14:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974869+0000", 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974870+0000", 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974876+0000", 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974889+0000", 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2670000000000001e-05 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974896+0000", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9370000000000004e-06 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979031+0000", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041348749999999997 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979163+0000", 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000132142 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.979423+0000", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025945100000000001 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981009+0000", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001585801 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989179+0000", 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081698940000000005 2024-09-22T01:14:25.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989243+0000", 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4480000000000006e-05 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989294+0000", 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0742999999999999e-05 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:133 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454327835999997, 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484576000000001, 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 133 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974945+0000", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974946+0000", 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0499999999999999e-06 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974952+0000", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974964+0000", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2164000000000001e-05 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.974976+0000", 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1875999999999999e-05 2024-09-22T01:14:25.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981087+0000", 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061111660000000003 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981224+0000", 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013626900000000001 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.981437+0000", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021382899999999999 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985221+0000", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037832999999999999 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989306+0000", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004085506 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989379+0000", 2024-09-22T01:14:25.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2909000000000003e-05 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989429+0000", 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0377000000000002e-05 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454224600000003, 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018165239999999999, 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 134 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975048+0000", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975049+0000", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975179+0000", 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000129722 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975197+0000", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8113000000000001e-05 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975207+0000", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5100000000000004e-06 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982219+0000", 2024-09-22T01:14:25.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070121760000000002 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982352+0000", 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000133249 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.982617+0000", 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026466800000000002 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989070+0000", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064531299999999996 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.992931+0000", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003860774 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993092+0000", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016092499999999999 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993213+0000", 2024-09-22T01:14:25.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121836 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454048327999999, 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018294939, 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 135 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975224+0000", 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975225+0000", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0999999999999997e-07 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975231+0000", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7699999999999998e-06 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975243+0000", 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1644000000000001e-05 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975250+0000", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.503e-06 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983862+0000", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086120459999999999 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.983989+0000", 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012705299999999999 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.984185+0000", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019606499999999999 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989159+0000", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049732120000000003 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993238+0000", 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040793449999999998 2024-09-22T01:14:25.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993372+0000", 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013405200000000001 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993519+0000", 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014733899999999999 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.454007095000001, 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018936457, 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 136 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:25.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975266+0000", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8400000000000004e-07 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975272+0000", 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:14:25.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975284+0000", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2393e-05 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975292+0000", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.763e-06 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985342+0000", 2024-09-22T01:14:25.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010050026 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985551+0000", 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208755 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.985988+0000", 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043766699999999998 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989513+0000", 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003525233 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994036+0000", 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0045223279999999999 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994117+0000", 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1549000000000001e-05 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994202+0000", 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4735999999999996e-05 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.453951259, 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019072927, 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975321+0000", 2024-09-22T01:14:25.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975323+0000", 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0699999999999999e-06 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975483+0000", 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000160416 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975500+0000", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7127000000000001e-05 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975509+0000", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4399999999999994e-06 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987086+0000", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011576606999999999 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987414+0000", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032788700000000001 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.987806+0000", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039176100000000002 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.993673+0000", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058668970000000003 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994227+0000", 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055439900000000002 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994330+0000", 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010278 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994394+0000", 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4542999999999995e-05 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.453745777000002, 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019163698999999999, 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975527+0000", 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975528+0000", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.017e-06 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975534+0000", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0469999999999999e-06 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975547+0000", 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2806000000000001e-05 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975554+0000", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5669999999999997e-06 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989583+0000", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014028175 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.989829+0000", 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024677100000000001 2024-09-22T01:14:25.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.990235+0000", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00040547400000000001 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994418+0000", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041833579999999999 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994534+0000", 2024-09-22T01:14:25.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011541500000000001 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994623+0000", 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9703000000000005e-05 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994691+0000", 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7366000000000005e-05 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.453706126999997, 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021194425999999999, 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975567+0000", 2024-09-22T01:14:25.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1299999999999999e-07 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975577+0000", 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4669999999999993e-06 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975587+0000", 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975594+0000", 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1770000000000001e-06 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991392+0000", 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015798764 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991632+0000", 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023988199999999999 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.991978+0000", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034544099999999999 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996441+0000", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004463608 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996541+0000", 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9623000000000005e-05 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996644+0000", 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103416 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996761+0000", 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000116532 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.534 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.453667336999999, 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022802374, 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975605+0000", 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975606+0000", 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0299999999999998e-07 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975744+0000", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013788599999999999 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975767+0000", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3003e-05 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975781+0000", 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.383e-05 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994785+0000", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.019003995999999999 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.994930+0000", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000144683 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.995124+0000", 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019457099999999999 2024-09-22T01:14:25.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998089+0000", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029647440000000001 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998221+0000", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013178199999999999 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998301+0000", 2024-09-22T01:14:25.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9839999999999995e-05 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998408+0000", 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107236 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.453475935, 2024-09-22T01:14:25.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024814201000000001, 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975797+0000", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975798+0000", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7000000000000003e-07 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975807+0000", 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.533e-06 2024-09-22T01:14:25.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975817+0000", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9599999999999995e-06 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.975824+0000", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3289999999999996e-06 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.996882+0000", 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021058038000000001 2024-09-22T01:14:25.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997106+0000", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022396800000000001 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.997464+0000", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035745699999999999 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.998789+0000", 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001325384 2024-09-22T01:14:25.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000443+0000", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001653493 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000521+0000", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:52.000611+0000", 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9662999999999999e-05 2024-09-22T01:14:25.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:25.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:25.546 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:25.547 INFO:teuthology.orchestra.run.smithi096.stdout:1239 writesame 0xe33f5ea thru 0xe34ac27 data_size 0x1(0xb63e bytes) 2024-09-22T01:14:25.547 INFO:teuthology.orchestra.run.smithi096.stdout:1240 writesame 0x21f6834 thru 0x22038e0 data_size 0x1(0xd0ad bytes) 2024-09-22T01:14:25.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1241 write 0xdf27d02 thru 0xdf2ab57 (0x2e56 bytes) 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1242 read 0xb9f35e0 thru 0xb9f7938 (0x4359 bytes) 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1243 write 0x6617258 thru 0x6625adc (0xe885 bytes) 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1252 write 0xc4d13e6 thru 0xc4d73f3 (0x600e bytes) 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1255 write 0x997d30a thru 0x998bde7 (0xeade bytes) 2024-09-22T01:14:25.549 INFO:teuthology.orchestra.run.smithi096.stdout:1256 trunc from 0xe34ac28 to 0x1976d29 2024-09-22T01:14:25.549 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:25.547+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:25.599 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.599 INFO:teuthology.orchestra.run.smithi096.stdout:1258 read 0x334261 thru 0x3343f6 (0x196 bytes) 2024-09-22T01:14:25.599 INFO:teuthology.orchestra.run.smithi096.stdout:1260 punch from 0x167ffe3 to 0x168a2ab, (0xa2c8 bytes) 2024-09-22T01:14:25.605 INFO:teuthology.orchestra.run.smithi096.stdout:1261 trunc from 0x1976d29 to 0x5d45aa7 2024-09-22T01:14:25.614 INFO:teuthology.orchestra.run.smithi096.stdout:1263 write 0x20171b8 thru 0x20206b1 (0x94fa bytes) 2024-09-22T01:14:25.617 INFO:teuthology.orchestra.run.smithi096.stdout:1264 read 0x1d38612 thru 0x1d3aa8a (0x2479 bytes) 2024-09-22T01:14:25.617 INFO:teuthology.orchestra.run.smithi096.stdout:1265 punch from 0x5323d54 to 0x532578d, (0x1a39 bytes) 2024-09-22T01:14:25.624 INFO:teuthology.orchestra.run.smithi096.stdout:1266 trunc from 0x5d45aa7 to 0xb5c5434 2024-09-22T01:14:25.645 INFO:teuthology.orchestra.run.smithi096.stdout:1268 trunc from 0xb5c5434 to 0x23f37fa 2024-09-22T01:14:25.648 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:25.647+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:14:25.689 INFO:teuthology.orchestra.run.smithi096.stdout:1269 write 0xdf8c3e5 thru 0xdf8c699 (0x2b5 bytes) 2024-09-22T01:14:25.698 INFO:teuthology.orchestra.run.smithi096.stdout:1270 trunc from 0xdf8c69a to 0xe0fe817 2024-09-22T01:14:25.710 INFO:teuthology.orchestra.run.smithi096.stdout:1271 read 0x4f0392f thru 0x4f0d329 (0x99fb bytes) 2024-09-22T01:14:25.710 INFO:teuthology.orchestra.run.smithi096.stdout:1272 writesame 0x84332f0 thru 0x84399b3 data_size 0x2(0x66c4 bytes) 2024-09-22T01:14:25.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:14:25.716 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:25.716 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:25.716 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:25.732 INFO:teuthology.orchestra.run.smithi096.stdout:1273 trunc from 0xe0fe817 to 0x482192a 2024-09-22T01:14:25.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:25.748 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:25.749+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:25.749 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.761 INFO:teuthology.orchestra.run.smithi096.stdout:1275 write 0x70dfaca thru 0x70ec778 (0xccaf bytes) 2024-09-22T01:14:25.770 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.770 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.770 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.770 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.770 INFO:teuthology.orchestra.run.smithi096.stdout:1282 write 0x2a9a818 thru 0x2a9d2b0 (0x2a99 bytes) 2024-09-22T01:14:25.773 INFO:teuthology.orchestra.run.smithi096.stdout:1283 writesame 0x3487b2d thru 0x348ff69 data_size 0x1(0x843d bytes) 2024-09-22T01:14:25.805 INFO:teuthology.orchestra.run.smithi096.stdout:1284 trunc from 0x70ec779 to 0x896abaa 2024-09-22T01:14:25.830 INFO:teuthology.orchestra.run.smithi096.stdout:1285 writesame 0xaffd7c0 thru 0xb009ff9 data_size 0x3(0xc83a bytes) 2024-09-22T01:14:25.830 INFO:teuthology.orchestra.run.smithi096.stdout:1286 read 0x88a79e thru 0x8983fe (0xdc61 bytes) 2024-09-22T01:14:25.830 INFO:teuthology.orchestra.run.smithi096.stdout:1287 write 0x2558cac thru 0x255e2ff (0x5654 bytes) 2024-09-22T01:14:25.846 INFO:teuthology.orchestra.run.smithi096.stdout:1288 trunc from 0xb009ffa to 0x25064ee 2024-09-22T01:14:25.849 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:25.847+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:25.873 INFO:teuthology.orchestra.run.smithi096.stdout:1289 read 0xeca61e thru 0xed8ffc (0xe9df bytes) 2024-09-22T01:14:25.874 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.874 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:25.885 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:25.885 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:25.886 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:25.886 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:25.886 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:25.893 INFO:teuthology.orchestra.run.smithi096.stdout:1295 write 0x97fe900 thru 0x980456d (0x5c6e bytes) 2024-09-22T01:14:25.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:25.911 INFO:teuthology.orchestra.run.smithi096.stdout:1297 write 0x9a1e746 thru 0x9a21ee2 (0x379d bytes) 2024-09-22T01:14:25.920 INFO:teuthology.orchestra.run.smithi096.stdout:1298 read 0x662aac4 thru 0x6633dd8 (0x9315 bytes) 2024-09-22T01:14:25.928 INFO:teuthology.orchestra.run.smithi096.stdout:1299 trunc from 0x9a21ee3 to 0x2af0219 2024-09-22T01:14:25.950 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:25.949+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:25.961 INFO:teuthology.orchestra.run.smithi096.stdout:1300 trunc from 0x2af0219 to 0xac7dc69 2024-09-22T01:14:25.970 INFO:teuthology.orchestra.run.smithi096.stdout:1301 clone 11 order 23 su 32768 sc 7 2024-09-22T01:14:26.032 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:26.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1738 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 131072~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.024459232, 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014237117000000001, 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1738 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006357+0000", 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5000999999999999e-05 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006366+0000", 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006372+0000", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8379999999999998e-06 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013432+0000", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070601079999999998 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013499+0000", 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7531000000000001e-05 2024-09-22T01:14:26.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020579+0000", 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070795700000000003 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2026 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3244032~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023747908000001, 2024-09-22T01:14:26.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014009057, 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2026 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:26.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007053+0000", 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007056+0000", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2649999999999999e-06 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007062+0000", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5570000000000001e-06 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007067+0000", 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8029999999999996e-06 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020933+0000", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013865475 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020999+0000", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390999999999995e-05 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021062+0000", 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-22T01:14:26.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2027 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023728113000001, 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014158201, 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2027 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007074+0000", 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007083+0000", 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9889999999999998e-06 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007089+0000", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5259999999999999e-06 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007094+0000", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6530000000000002e-06 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021090+0000", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01399624 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021157+0000", 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6538999999999994e-05 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021231+0000", 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4640000000000004e-05 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2028 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3801088~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023700596999999, 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014289873, 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2028 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007101+0000", 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:26.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007103+0000", 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007111+0000", 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007122+0000", 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988000000000001e-05 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021258+0000", 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014136319 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021326+0000", 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7642000000000001e-05 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021390+0000", 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4645000000000002e-05 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023672523, 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:26.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:26.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023651854000001, 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:26.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023621541000001, 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023602286999999, 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:26.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023584185000001, 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:26.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:26.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023563462, 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:26.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023542973, 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:26.078 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:26.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023521531, 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:26.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:26.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023490684, 2024-09-22T01:14:26.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:26.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023453158000001, 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:26.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:26.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:26.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.02342103, 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:26.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.023398023, 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.105 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.984084190000001, 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:26.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:26.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982797010000001, 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.115 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982452826999999, 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:26.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:26.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:26.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:15337 3.e 3:73ab052b:::rbd_data.1124645eb891.0000000000000038:c [sparse-read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.48459329800000001, 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017128462000000001, 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15337 2024-09-22T01:14:26.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546210+0000", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546215+0000", 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3800000000000002e-06 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546219+0000", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0620000000000002e-06 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554742+0000", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085230519999999997 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554860+0000", 2024-09-22T01:14:26.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117883 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.563336+0000", 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084759050000000006 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.126 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:26.128 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:26.128 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:26.129+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:26.229 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:26.229+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:26.291 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:26.291 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:26.291 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:26.291 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:26.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:26.330 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:26.329+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:26.422 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:26.467 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:26.467+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:26.531 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:26.533+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:26.543 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.523747086, 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:14:26.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:14:26.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2200 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7110656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519181538, 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046602256000000002, 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2200 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:26.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024422+0000", 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024429+0000", 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024434+0000", 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070876+0000", 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046442215000000002 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070950+0000", 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3189999999999996e-05 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071022+0000", 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2273000000000004e-05 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2201 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519159044, 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046753765000000003, 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2201 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024445+0000", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3489999999999999e-06 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024451+0000", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4470000000000003e-06 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024457+0000", 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2024-09-22T01:14:26.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071061+0000", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046603415000000002 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071123+0000", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2509999999999996e-05 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071196+0000", 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2719000000000006e-05 2024-09-22T01:14:26.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2202 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519136049, 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046876332, 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2202 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:26.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024468+0000", 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024474+0000", 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024480+0000", 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071230+0000", 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046749717000000003 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071298+0000", 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8266e-05 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071341+0000", 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3350000000000003e-05 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.564 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2203 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519113826, 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046984537999999999, 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2203 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024488+0000", 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024490+0000", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024497+0000", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9970000000000001e-06 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024503+0000", 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.31e-06 2024-09-22T01:14:26.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071374+0000", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046871082000000001 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071423+0000", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9172999999999998e-05 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071472+0000", 2024-09-22T01:14:26.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8973e-05 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2211 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519091221, 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124055999999998, 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2211 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:26.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3399999999999999e-07 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024513+0000", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024520+0000", 2024-09-22T01:14:26.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7410000000000003e-06 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024533+0000", 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2867999999999999e-05 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071509+0000", 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046976047 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071569+0000", 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0482999999999997e-05 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071634+0000", 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4834999999999998e-05 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2212 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:26.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519060388, 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047215269999999997, 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2212 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3e-07 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024544+0000", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0690000000000002e-06 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024551+0000", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.686e-06 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024557+0000", 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.806e-06 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071661+0000", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047104241999999998 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071703+0000", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071756+0000", 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3171999999999998e-05 2024-09-22T01:14:26.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2213 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.519025293, 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047299709000000002, 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2213 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024593+0000", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6223e-05 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024600+0000", 2024-09-22T01:14:26.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024607+0000", 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080000000000003e-06 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071783+0000", 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047175863999999998 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071822+0000", 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071875+0000", 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3801999999999998e-05 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2214 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.582 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.518985416, 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047393246999999999, 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2214 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026492+0000", 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018767059999999999 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026500+0000", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026513+0000", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3422e-05 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026523+0000", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071904+0000", 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045380993000000001 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071942+0000", 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572999999999998e-05 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.072009+0000", 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6797000000000004e-05 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.586 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2405 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 4915200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514462515, 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04671264, 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2405 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029140+0000", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1209999999999999e-06 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029143+0000", 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2049999999999998e-06 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029146+0000", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7800000000000001e-06 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075719+0000", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046573497999999998 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075787+0000", 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7135e-05 2024-09-22T01:14:26.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075851+0000", 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4660999999999996e-05 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2406 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514451572, 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046865920999999998, 2024-09-22T01:14:26.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2406 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.22e-07 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029151+0000", 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:14:26.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029154+0000", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.439e-06 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029158+0000", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075881+0000", 2024-09-22T01:14:26.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046723187999999999 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075951+0000", 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9565000000000002e-05 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076015+0000", 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4931000000000006e-05 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2407 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514439357000001, 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047019894, 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2407 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3900000000000001e-07 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029163+0000", 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2190000000000001e-06 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029167+0000", 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3390000000000001e-06 2024-09-22T01:14:26.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029169+0000", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9249999999999999e-06 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076045+0000", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046875173999999999 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076114+0000", 2024-09-22T01:14:26.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.902e-05 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076182+0000", 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7978000000000005e-05 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2408 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:26.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514427854999999, 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047176774999999997, 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2408 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:26.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.35e-07 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029175+0000", 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029178+0000", 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2650000000000001e-06 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029181+0000", 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6860000000000002e-06 2024-09-22T01:14:26.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076216+0000", 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035615000000003 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076286+0000", 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9680999999999999e-05 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076350+0000", 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4172999999999997e-05 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2409 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514416859000001, 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047331204000000002, 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2409 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-07 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029186+0000", 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029190+0000", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1169999999999997e-06 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029194+0000", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5310000000000003e-06 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076377+0000", 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047182737000000002 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076447+0000", 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0411000000000001e-05 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076515+0000", 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7960999999999995e-05 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2410 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514384788999999, 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047479322999999997, 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2410 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029217+0000", 2024-09-22T01:14:26.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029219+0000", 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.339e-06 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029226+0000", 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6429999999999998e-06 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029231+0000", 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.152e-06 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076544+0000", 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047312694000000002 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076630+0000", 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5753000000000004e-05 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076696+0000", 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6087999999999995e-05 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2411 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514355467, 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047606067000000002, 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2411 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029249+0000", 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4739999999999999e-06 2024-09-22T01:14:26.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029256+0000", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999996e-06 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029260+0000", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076724+0000", 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047463678000000002 2024-09-22T01:14:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076786+0000", 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000005e-05 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076852+0000", 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5759999999999994e-05 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2412 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514333331, 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047679747000000001, 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2412 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029270+0000", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0540000000000002e-06 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029277+0000", 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029282+0000", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6890000000000003e-06 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076869+0000", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047587741000000003 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076909+0000", 2024-09-22T01:14:26.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9504e-05 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076947+0000", 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8578999999999998e-05 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2413 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514311233000001, 2024-09-22T01:14:26.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047815037999999997, 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2413 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029292+0000", 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:14:26.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029299+0000", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029304+0000", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076974+0000", 2024-09-22T01:14:26.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047670055000000003 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077036+0000", 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2019000000000004e-05 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077105+0000", 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9127000000000004e-05 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2414 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.514290503, 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048254547000000002, 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2414 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:26.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029313+0000", 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.075e-06 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029320+0000", 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029325+0000", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077133+0000", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047808311999999999 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077197+0000", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3912999999999996e-05 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077565+0000", 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830000000000001 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3931 3.11 3:8f505f18:::rbd_data.1124ef2748ce.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.428422308, 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048441529999999997, 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:26.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3931 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:26.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115182+0000", 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:26.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115189+0000", 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115194+0000", 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160040+0000", 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044845812999999998 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160142+0000", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010177499999999999 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160287+0000", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014540899999999999 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163457+0000", 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031691570000000001 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163508+0000", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1944000000000003e-05 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163581+0000", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2123e-05 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163620+0000", 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709000000000002e-05 2024-09-22T01:14:26.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:26.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:26.638 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:26.639 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:26.641+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:26.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:26.710 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:14:26.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:14:26.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:14:26.777 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:26.777 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:26.777 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:26.777 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:26.779 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:26.779+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:26.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:26.815 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone10 from 0xac7dc69 (overlap 0x1976d29) to 0x16eaf0b 2024-09-22T01:14:26.840 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:26.839+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:26.857 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:26.858 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:26.884 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:26.902 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:26.902 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:26.921 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:26.921 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:26.922 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:26.922 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:26.923 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:26.923 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:26.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-22T01:14:26.940 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:26.939+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:26.943 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:26.943 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:26.961 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:26.961 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:26.979 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:26.979 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:26.997 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:26.997 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:27.041 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:27.041+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:27.058 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.059 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.073 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:27.142 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:27.139+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:27.232 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.233 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.243 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:27.245+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:27.247 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:27.321 INFO:teuthology.orchestra.run.smithi096.stdout:1302 read 0x1c5a843 thru 0x1c5e831 (0x3fef bytes) 2024-09-22T01:14:27.323 INFO:teuthology.orchestra.run.smithi096.stdout:1303 read 0x7d33880 thru 0x7d42a13 (0xf194 bytes) 2024-09-22T01:14:27.340 INFO:teuthology.orchestra.run.smithi096.stdout:1304 write 0xb9adff3 thru 0xb9b9d6b (0xbd79 bytes) 2024-09-22T01:14:27.344 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:27.345+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:27.350 INFO:teuthology.orchestra.run.smithi096.stdout:1305 write 0x75cb96 thru 0x76a1a1 (0xd60c bytes) 2024-09-22T01:14:27.420 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.421 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.443 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:27.462 INFO:teuthology.orchestra.run.smithi096.stdout:1306 trunc from 0xb9b9d6c to 0x260303f 2024-09-22T01:14:27.485 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:27.485+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:27.546 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:27.545+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:27.606 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.607 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:27.625 INFO:teuthology.orchestra.run.smithi096.stdout:1309 write 0xd38de6d thru 0xd39c0c4 (0xe258 bytes) 2024-09-22T01:14:27.636 INFO:teuthology.orchestra.run.smithi096.stdout:1310 writesame 0x6280b56 thru 0x628485f data_size 0xd(0x3d0a bytes) 2024-09-22T01:14:27.643 INFO:teuthology.orchestra.run.smithi096.stdout:1313 writesame 0x4446cbc thru 0x444bde2 data_size 0x19(0x5127 bytes) 2024-09-22T01:14:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:1317 read 0x87788a8 thru 0x877e315 (0x5a6e bytes) 2024-09-22T01:14:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:1318 writesame 0x63176c9 thru 0x631f739 data_size 0x1(0x8071 bytes) 2024-09-22T01:14:27.664 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:27.665+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:27.707 INFO:teuthology.orchestra.run.smithi096.stdout:1320 read 0x84d1818 thru 0x84dcc78 (0xb461 bytes) 2024-09-22T01:14:27.707 INFO:teuthology.orchestra.run.smithi096.stdout:1321 punch from 0x71bd85e to 0x71c26b8, (0x4e5a bytes) 2024-09-22T01:14:27.707 INFO:teuthology.orchestra.run.smithi096.stdout:1323 punch from 0x2ddc2cc to 0x2de1812, (0x5546 bytes) 2024-09-22T01:14:27.707 INFO:teuthology.orchestra.run.smithi096.stdout:1325 writesame 0xa1ab12 thru 0xa1dd8f data_size 0x17(0x327e bytes) 2024-09-22T01:14:27.707 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:14:27.712 INFO:teuthology.orchestra.run.smithi096.stdout:1326 writesame 0x7fd717c thru 0x7fe551f data_size 0x22(0xe3a4 bytes) 2024-09-22T01:14:27.715 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:14:27.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:14:27.739 INFO:teuthology.orchestra.run.smithi096.stdout:1327 writesame 0xd976f35 thru 0xd97aec0 data_size 0x1c(0x3f8c bytes) 2024-09-22T01:14:27.741 INFO:teuthology.orchestra.run.smithi096.stdout:1330 read 0x98fc1f3 thru 0x98fdb43 (0x1951 bytes) 2024-09-22T01:14:27.741 INFO:teuthology.orchestra.run.smithi096.stdout:1331 write 0x1c6f8e0 thru 0x1c7a6f0 (0xae11 bytes) 2024-09-22T01:14:27.744 INFO:teuthology.orchestra.run.smithi096.stdout:1332 read 0xedcd63 thru 0xee42bd (0x755b bytes) 2024-09-22T01:14:27.745 INFO:teuthology.orchestra.run.smithi096.stdout:1333 punch from 0x3630e3 to 0x36a8a1, (0x77be bytes) 2024-09-22T01:14:27.745 INFO:teuthology.orchestra.run.smithi096.stdout:1334 write 0x17fac52 thru 0x1809cda (0xf089 bytes) 2024-09-22T01:14:27.747 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:27.747+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:27.759 INFO:teuthology.orchestra.run.smithi096.stdout:1335 write 0xecac1fa thru 0xecb8972 (0xc779 bytes) 2024-09-22T01:14:27.771 INFO:teuthology.orchestra.run.smithi096.stdout:1336 trunc from 0xecb8973 to 0xc558705 2024-09-22T01:14:27.789 INFO:teuthology.orchestra.run.smithi096.stdout:1337 read 0x4b6570d thru 0x4b6fae0 (0xa3d4 bytes) 2024-09-22T01:14:27.789 INFO:teuthology.orchestra.run.smithi096.stdout:1339 read 0x9cf5877 thru 0x9cf9ac9 (0x4253 bytes) 2024-09-22T01:14:27.789 INFO:teuthology.orchestra.run.smithi096.stdout:1340 punch from 0x6ea22dd to 0x6ead622, (0xb345 bytes) 2024-09-22T01:14:27.789 INFO:teuthology.orchestra.run.smithi096.stdout:1341 writesame 0x4551d5d thru 0x455cf42 data_size 0x7(0xb1e6 bytes) 2024-09-22T01:14:27.791 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.791 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.796 INFO:teuthology.orchestra.run.smithi096.stdout:1342 trunc from 0xc558705 to 0xc4bc607 2024-09-22T01:14:27.807 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:27.808 INFO:teuthology.orchestra.run.smithi096.stdout:1343 write 0x320dc1d thru 0x3219c7b (0xc05f bytes) 2024-09-22T01:14:27.815 INFO:teuthology.orchestra.run.smithi096.stdout:1344 trunc from 0xc4bc607 to 0x867b9e6 2024-09-22T01:14:27.832 INFO:teuthology.orchestra.run.smithi096.stdout:1345 writesame 0x17b9cd6 thru 0x17bc1b1 data_size 0xe(0x24dc bytes) 2024-09-22T01:14:27.834 INFO:teuthology.orchestra.run.smithi096.stdout:1347 write 0x50ee931 thru 0x50f8cbe (0xa38e bytes) 2024-09-22T01:14:27.837 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:27.841 INFO:teuthology.orchestra.run.smithi096.stdout:1350 trunc from 0x867b9e6 to 0x642af0d 2024-09-22T01:14:27.848 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:27.847+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:27.871 INFO:teuthology.orchestra.run.smithi096.stdout:1351 write 0x7bdc43c thru 0x7be4a28 (0x85ed bytes) 2024-09-22T01:14:27.881 INFO:teuthology.orchestra.run.smithi096.stdout:1352 read 0x6627cce thru 0x6628606 (0x939 bytes) 2024-09-22T01:14:27.881 INFO:teuthology.orchestra.run.smithi096.stdout:1353 write 0x6f2af80 thru 0x6f2b1b5 (0x236 bytes) 2024-09-22T01:14:27.883 INFO:teuthology.orchestra.run.smithi096.stdout:1354 read 0x4089ffa thru 0x408d8bd (0x38c4 bytes) 2024-09-22T01:14:27.886 INFO:teuthology.orchestra.run.smithi096.stdout:1355 trunc from 0x7be4a29 to 0x9590371 2024-09-22T01:14:27.893 INFO:teuthology.orchestra.run.smithi096.stdout:1356 read 0x368e90a thru 0x3692cd4 (0x43cb bytes) 2024-09-22T01:14:27.894 INFO:teuthology.orchestra.run.smithi096.stdout:1357 writesame 0x34d7871 thru 0x34d9b9f data_size 0x1(0x232f bytes) 2024-09-22T01:14:27.915 INFO:teuthology.orchestra.run.smithi096.stdout:1358 writesame 0xd81e995 thru 0xd82e98a data_size 0xa3(0xfff6 bytes) 2024-09-22T01:14:27.922 INFO:teuthology.orchestra.run.smithi096.stdout:1359 trunc from 0xd82e98b to 0x91196da 2024-09-22T01:14:27.930 INFO:teuthology.orchestra.run.smithi096.stdout:1360 writesame 0x567ac8d thru 0x567f3c6 data_size 0x9(0x473a bytes) 2024-09-22T01:14:27.930 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:27.930 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:1361 punch from 0x6e4747c to 0x6e47819, (0x39d bytes) 2024-09-22T01:14:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:1362 read 0x86598e2 thru 0x8668be5 (0xf304 bytes) 2024-09-22T01:14:27.933 INFO:teuthology.orchestra.run.smithi096.stdout:1363 write 0x118f9b3 thru 0x1199b34 (0xa182 bytes) 2024-09-22T01:14:27.939 INFO:teuthology.orchestra.run.smithi096.stdout:1364 writesame 0x420fe24 thru 0x42143fd data_size 0x1(0x45da bytes) 2024-09-22T01:14:27.946 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:27.948 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:27.949+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:27.953 INFO:teuthology.orchestra.run.smithi096.stdout:1365 read 0x32696a thru 0x33410f (0xd7a6 bytes) 2024-09-22T01:14:27.954 INFO:teuthology.orchestra.run.smithi096.stdout:1366 writesame 0x4f2c018 thru 0x4f32645 data_size 0x3a(0x662e bytes) 2024-09-22T01:14:27.969 INFO:teuthology.orchestra.run.smithi096.stdout:1367 writesame 0xab796c7 thru 0xab84db3 data_size 0x1(0xb6ed bytes) 2024-09-22T01:14:27.969 INFO:teuthology.orchestra.run.smithi096.stdout:1368 read 0x822449d thru 0x822fa43 (0xb5a7 bytes) 2024-09-22T01:14:27.969 INFO:teuthology.orchestra.run.smithi096.stdout:1369 write 0x3ff3ddb thru 0x3ffab9c (0x6dc2 bytes) 2024-09-22T01:14:27.991 INFO:teuthology.orchestra.run.smithi096.stdout:1370 writesame 0xe1e103a thru 0xe1e132a data_size 0x1(0x2f1 bytes) 2024-09-22T01:14:27.991 INFO:teuthology.orchestra.run.smithi096.stdout:1371 write 0x1da8922 thru 0x1dae10d (0x57ec bytes) 2024-09-22T01:14:27.994 INFO:teuthology.orchestra.run.smithi096.stdout:1372 writesame 0x798e68c thru 0x79972de data_size 0x1(0x8c53 bytes) 2024-09-22T01:14:28.044 INFO:teuthology.orchestra.run.smithi096.stdout:1376 trunc from 0xe1e132b to 0x6c2f9fc 2024-09-22T01:14:28.050 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:28.049+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:28.070 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.070 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.085 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:28.087 INFO:teuthology.orchestra.run.smithi096.stdout:1377 write 0xadc1352 thru 0xadc13c1 (0x70 bytes) 2024-09-22T01:14:28.093 INFO:teuthology.orchestra.run.smithi096.stdout:1378 read 0x8006879 thru 0x8011f1d (0xb6a5 bytes) 2024-09-22T01:14:28.093 INFO:teuthology.orchestra.run.smithi096.stdout:1379 writesame 0x117fc66 thru 0x1180a37 data_size 0x2(0xdd2 bytes) 2024-09-22T01:14:28.098 INFO:teuthology.orchestra.run.smithi096.stdout:1380 write 0x89e0ffb thru 0x89e9d39 (0x8d3f bytes) 2024-09-22T01:14:28.103 INFO:teuthology.orchestra.run.smithi096.stdout:1381 write 0x3777c55 thru 0x377bdc6 (0x4172 bytes) 2024-09-22T01:14:28.106 INFO:teuthology.orchestra.run.smithi096.stdout:1382 write 0x9499adf thru 0x94a10ab (0x75cd bytes) 2024-09-22T01:14:28.108 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.127 INFO:teuthology.orchestra.run.smithi096.stdout:1384 write 0xe8d3f55 thru 0xe8dcb0e (0x8bba bytes) 2024-09-22T01:14:28.136 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.136 INFO:teuthology.orchestra.run.smithi096.stdout:1386 punch from 0x6bf5832 to 0x6bfaf9c, (0x576a bytes) 2024-09-22T01:14:28.136 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.136 INFO:teuthology.orchestra.run.smithi096.stdout:1388 write 0x1d739cf thru 0x1d76608 (0x2c3a bytes) 2024-09-22T01:14:28.138 INFO:teuthology.orchestra.run.smithi096.stdout:1390 write 0x5b3f060 thru 0x5b425e0 (0x3581 bytes) 2024-09-22T01:14:28.141 INFO:teuthology.orchestra.run.smithi096.stdout:1392 write 0x3f9905 thru 0x4039c5 (0xa0c1 bytes) 2024-09-22T01:14:28.144 INFO:teuthology.orchestra.run.smithi096.stdout:1393 write 0x61f33c6 thru 0x6202dfb (0xfa36 bytes) 2024-09-22T01:14:28.149 INFO:teuthology.orchestra.run.smithi096.stdout:1394 read 0x47f405 thru 0x481216 (0x1e12 bytes) 2024-09-22T01:14:28.150 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.150 INFO:teuthology.orchestra.run.smithi096.stdout:1396 read 0x8c65916 thru 0x8c74db4 (0xf49f bytes) 2024-09-22T01:14:28.150 INFO:teuthology.orchestra.run.smithi096.stdout:1397 write 0xb8c101b thru 0xb8c8603 (0x75e9 bytes) 2024-09-22T01:14:28.156 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.157 INFO:teuthology.orchestra.run.smithi096.stdout:1400 punch from 0xa0c07ec to 0xa0cc80e, (0xc022 bytes) 2024-09-22T01:14:28.157 INFO:teuthology.orchestra.run.smithi096.stdout:1402 read 0x3a4a555 thru 0x3a4ebe4 (0x4690 bytes) 2024-09-22T01:14:28.157 INFO:teuthology.orchestra.run.smithi096.stdout:1403 write 0xd226f50 thru 0xd227d18 (0xdc9 bytes) 2024-09-22T01:14:28.159 INFO:teuthology.orchestra.run.smithi096.stdout:1404 read 0x764188b thru 0x7642eb4 (0x162a bytes) 2024-09-22T01:14:28.160 INFO:teuthology.orchestra.run.smithi096.stdout:1406 writesame 0xdf3dbf7 thru 0xdf4cedd data_size 0x1(0xf2e7 bytes) 2024-09-22T01:14:28.171 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:28.173+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:28.251 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:28.249+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:28.253 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.253 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.261 INFO:teuthology.orchestra.run.smithi096.stdout:1407 write 0xc9a8b33 thru 0xc9b54c5 (0xc993 bytes) 2024-09-22T01:14:28.268 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:28.301 INFO:teuthology.orchestra.run.smithi096.stdout:1408 trunc from 0xe8dcb0f to 0x306e592 2024-09-22T01:14:28.343 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.343 INFO:teuthology.orchestra.run.smithi096.stdout:1410 read 0x1c20920 thru 0x1c2aacc (0xa1ad bytes) 2024-09-22T01:14:28.355 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:28.357+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:28.384 INFO:teuthology.orchestra.run.smithi096.stdout:1411 write 0xe3147f0 thru 0xe323ebf (0xf6d0 bytes) 2024-09-22T01:14:28.396 INFO:teuthology.orchestra.run.smithi096.stdout:1413 read 0x7cac9da thru 0x7cae345 (0x196c bytes) 2024-09-22T01:14:28.396 INFO:teuthology.orchestra.run.smithi096.stdout:1414 write 0x1b358b3 thru 0x1b3ff1f (0xa66d bytes) 2024-09-22T01:14:28.400 INFO:teuthology.orchestra.run.smithi096.stdout:1417 punch from 0xd05a011 to 0xd05f19e, (0x518d bytes) 2024-09-22T01:14:28.400 INFO:teuthology.orchestra.run.smithi096.stdout:1419 read 0x85ab8c9 thru 0x85ad762 (0x1e9a bytes) 2024-09-22T01:14:28.400 INFO:teuthology.orchestra.run.smithi096.stdout:1420 trunc from 0xe323ec0 to 0xe308f59 2024-09-22T01:14:28.416 INFO:teuthology.orchestra.run.smithi096.stdout:1421 write 0xc41d737 thru 0xc426b9b (0x9465 bytes) 2024-09-22T01:14:28.427 INFO:teuthology.orchestra.run.smithi096.stdout:1422 read 0x48f0d86 thru 0x48fba73 (0xacee bytes) 2024-09-22T01:14:28.427 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.427 INFO:teuthology.orchestra.run.smithi096.stdout:1424 write 0x7c16541 thru 0x7c1f895 (0x9355 bytes) 2024-09-22T01:14:28.437 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:28.437 INFO:teuthology.orchestra.run.smithi096.stdout:1426 read 0x1c66cf3 thru 0x1c6f63d (0x894b bytes) 2024-09-22T01:14:28.439 INFO:teuthology.orchestra.run.smithi096.stdout:1427 punch from 0x3d42c49 to 0x3d52a6c, (0xfe23 bytes) 2024-09-22T01:14:28.439 INFO:teuthology.orchestra.run.smithi096.stdout:1428 write 0x36a07b8 thru 0x36a830f (0x7b58 bytes) 2024-09-22T01:14:28.442 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.442 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.453 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:28.453+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:28.456 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:28.463 INFO:teuthology.orchestra.run.smithi096.stdout:1430 trunc from 0xe308f59 to 0x60d88e1 2024-09-22T01:14:28.488 INFO:teuthology.orchestra.run.smithi096.stdout:1432 clone 12 order 24 su 262144 sc 9 2024-09-22T01:14:28.554 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:28.553+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:28.629 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.629 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:28.687 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:28.687+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:14:28.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:14:28.787 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:28.787+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:28.788 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.789 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:28.830 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone11 from 0x60d88e1 (overlap 0x260303f) to 0x1e68cff 2024-09-22T01:14:28.879 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:28.879+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:28.955 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:28.956 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:28.958 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:28.955+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:28.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:28.983 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-22T01:14:29.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:29.032 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:29.049 INFO:teuthology.orchestra.run.smithi096.stdout:1434 write 0x636b894 thru 0x636fd71 (0x44de bytes) 2024-09-22T01:14:29.059 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:29.059+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:29.114 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:29.114 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:29.122 INFO:teuthology.orchestra.run.smithi096.stdout:1436 write 0xaaf2368 thru 0xaafb3ce (0x9067 bytes) 2024-09-22T01:14:29.127 INFO:teuthology.orchestra.run.smithi096.stdout:1439 write 0x2f4fc2e thru 0x2f5e579 (0xe94c bytes) 2024-09-22T01:14:29.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:29.160 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:29.161+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:29.174 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:29.174 INFO:teuthology.orchestra.run.smithi096.stdout:1441 write 0x27ce6d3 thru 0x27d892d (0xa25b bytes) 2024-09-22T01:14:29.190 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:29.190 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:29.210 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:29.210 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:29.224 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:29.227 INFO:teuthology.orchestra.run.smithi096.stdout:1443 trunc from 0xaafb3cf to 0xc91df0e 2024-09-22T01:14:29.231 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:29.231 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:29.237 INFO:teuthology.orchestra.run.smithi096.stdout:1444 write 0x1e79edb thru 0x1e7cac6 (0x2bec bytes) 2024-09-22T01:14:29.256 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:29.257 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:29.260 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:29.259+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:29.280 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:29.280 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:29.300 INFO:teuthology.orchestra.run.smithi096.stdout:1446 writesame 0xa3b136e thru 0xa3b6f34 data_size 0x25(0x5bc7 bytes) 2024-09-22T01:14:29.302 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:29.302 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:29.307 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:29.307 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:29.320 INFO:teuthology.orchestra.run.smithi096.stdout:1448 trunc from 0xc91df0e to 0x113022b 2024-09-22T01:14:29.323 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:29.324 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:29.331 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:29.362 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:29.361+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:29.364 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:29.451 INFO:teuthology.orchestra.run.smithi096.stdout:1449 write 0x9551cdb thru 0x95588cd (0x6bf3 bytes) 2024-09-22T01:14:29.459 INFO:teuthology.orchestra.run.smithi096.stdout:1450 trunc from 0x95588ce to 0xbe51ee6 2024-09-22T01:14:29.462 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:29.459+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:29.474 INFO:teuthology.orchestra.run.smithi096.stdout:1451 writesame 0xca740ee thru 0xca7a19f data_size 0x1(0x60b2 bytes) 2024-09-22T01:14:29.474 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:29.474 INFO:teuthology.orchestra.run.smithi096.stdout:1453 read 0x6cfdc1b thru 0x6d05e75 (0x825b bytes) 2024-09-22T01:14:29.475 INFO:teuthology.orchestra.run.smithi096.stdout:1454 read 0xb723518 thru 0xb732b89 (0xf672 bytes) 2024-09-22T01:14:29.475 INFO:teuthology.orchestra.run.smithi096.stdout:1455 punch from 0xc835fde to 0xc84430b, (0xe32d bytes) 2024-09-22T01:14:29.475 INFO:teuthology.orchestra.run.smithi096.stdout:1456 read 0xc79fcce thru 0xc7af788 (0xfabb bytes) 2024-09-22T01:14:29.476 INFO:teuthology.orchestra.run.smithi096.stdout:1457 read 0x9050e8e thru 0x9060702 (0xf875 bytes) 2024-09-22T01:14:29.476 INFO:teuthology.orchestra.run.smithi096.stdout:1458 write 0x2daa413 thru 0x2dad736 (0x3324 bytes) 2024-09-22T01:14:29.478 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:29.478 INFO:teuthology.orchestra.run.smithi096.stdout:1460 write 0xaab788c thru 0xaabab85 (0x32fa bytes) 2024-09-22T01:14:29.484 INFO:teuthology.orchestra.run.smithi096.stdout:1461 read 0xb161123 thru 0xb1681b7 (0x7095 bytes) 2024-09-22T01:14:29.484 INFO:teuthology.orchestra.run.smithi096.stdout:1462 writesame 0x17b3678 thru 0x17c0964 data_size 0x29(0xd2ed bytes) 2024-09-22T01:14:29.488 INFO:teuthology.orchestra.run.smithi096.stdout:1464 read 0x863fa96 thru 0x8640e5a (0x13c5 bytes) 2024-09-22T01:14:29.496 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:29.496 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:29.497 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:29.497 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:29.497 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:29.505 INFO:teuthology.orchestra.run.smithi096.stdout:1465 trunc from 0xca7a1a0 to 0x1b0769c 2024-09-22T01:14:29.517 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:29.552 INFO:teuthology.orchestra.run.smithi096.stdout:1466 write 0x985e1bc thru 0x9867f2b (0x9d70 bytes) 2024-09-22T01:14:29.557 INFO:teuthology.orchestra.run.smithi096.stdout:1468 trunc from 0x9867f2c to 0xa7fe4c5 2024-09-22T01:14:29.564 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:29.565+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:29.567 INFO:teuthology.orchestra.run.smithi096.stdout:1469 read 0x3ad36be thru 0x3ada4f1 (0x6e34 bytes) 2024-09-22T01:14:29.577 INFO:teuthology.orchestra.run.smithi096.stdout:1472 write 0xa990eeb thru 0xa99d9a4 (0xcaba bytes) 2024-09-22T01:14:29.583 INFO:teuthology.orchestra.run.smithi096.stdout:1473 punch from 0x7bef6a4 to 0x7bf33f0, (0x3d4c bytes) 2024-09-22T01:14:29.583 INFO:teuthology.orchestra.run.smithi096.stdout:1474 read 0x75d4ee1 thru 0x75e1952 (0xca72 bytes) 2024-09-22T01:14:29.583 INFO:teuthology.orchestra.run.smithi096.stdout:1475 write 0x2c5d69b thru 0x2c6106e (0x39d4 bytes) 2024-09-22T01:14:29.585 INFO:teuthology.orchestra.run.smithi096.stdout:1476 writesame 0x8a3566e thru 0x8a4402e data_size 0x6d(0xe9c1 bytes) 2024-09-22T01:14:29.589 INFO:teuthology.orchestra.run.smithi096.stdout:1477 write 0x40b0ab1 thru 0x40b0ce2 (0x232 bytes) 2024-09-22T01:14:29.591 INFO:teuthology.orchestra.run.smithi096.stdout:1478 read 0x91554b8 thru 0x916115b (0xbca4 bytes) 2024-09-22T01:14:29.594 INFO:teuthology.orchestra.run.smithi096.stdout:1479 trunc from 0xa99d9a5 to 0xc8987d4 2024-09-22T01:14:29.604 INFO:teuthology.orchestra.run.smithi096.stdout:1481 read 0x8ec082 thru 0x8f19d2 (0x5951 bytes) 2024-09-22T01:14:29.605 INFO:teuthology.orchestra.run.smithi096.stdout:1482 write 0x2b7cac5 thru 0x2b8afbf (0xe4fb bytes) 2024-09-22T01:14:29.609 INFO:teuthology.orchestra.run.smithi096.stdout:1483 read 0xaee2509 thru 0xaeeeb8e (0xc686 bytes) 2024-09-22T01:14:29.609 INFO:teuthology.orchestra.run.smithi096.stdout:1484 write 0x8532dd2 thru 0x8536bdf (0x3e0e bytes) 2024-09-22T01:14:29.612 INFO:teuthology.orchestra.run.smithi096.stdout:1486 read 0xf2b69d thru 0xf30c5c (0x55c0 bytes) 2024-09-22T01:14:29.612 INFO:teuthology.orchestra.run.smithi096.stdout:1487 write 0xaad30b3 thru 0xaae2098 (0xefe6 bytes) 2024-09-22T01:14:29.619 INFO:teuthology.orchestra.run.smithi096.stdout:1489 write 0x5e3aaba thru 0x5e3c2d8 (0x181f bytes) 2024-09-22T01:14:29.622 INFO:teuthology.orchestra.run.smithi096.stdout:1490 write 0x860c67a thru 0x8617776 (0xb0fd bytes) 2024-09-22T01:14:29.627 INFO:teuthology.orchestra.run.smithi096.stdout:1491 read 0x57a0bef thru 0x57a2ff3 (0x2405 bytes) 2024-09-22T01:14:29.628 INFO:teuthology.orchestra.run.smithi096.stdout:1494 clone 13 order 23 su 4194304 sc 15 2024-09-22T01:14:29.665 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:29.663+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 38.695003514, 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:29.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:29.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20645 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34240909600000002, 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044103126999999999, 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20645 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337063+0000", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337066+0000", 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9050000000000001e-06 2024-09-22T01:14:29.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337077+0000", 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0693000000000001e-05 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337083+0000", 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2559999999999999e-06 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.380731+0000", 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043648570999999997 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.380790+0000", 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8174999999999998e-05 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381166+0000", 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000375981 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20646 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.342380407, 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044647025, 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20646 2024-09-22T01:14:29.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337092+0000", 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1500000000000004e-07 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337094+0000", 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2230000000000001e-06 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337101+0000", 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337105+0000", 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8389999999999997e-06 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381224+0000", 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044118460999999998 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381315+0000", 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1325999999999999e-05 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381738+0000", 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042295400000000002 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20647 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34235950300000001, 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045060006, 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20647 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999999e-07 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337115+0000", 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2500000000000001e-06 2024-09-22T01:14:29.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337123+0000", 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1790000000000008e-06 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337128+0000", 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7639999999999996e-06 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381789+0000", 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044661197 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381855+0000", 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6688999999999998e-05 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382172+0000", 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031653600000000002 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20648 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34233645899999998, 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045676618000000002, 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20648 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337136+0000", 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337145+0000", 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7540000000000001e-06 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337238+0000", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2487999999999999e-05 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337258+0000", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0245e-05 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382230+0000", 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044971934999999998 2024-09-22T01:14:29.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382345+0000", 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001155 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382812+0000", 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046620400000000001 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20649 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [sparse-read 5013504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34220171399999999, 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046375128000000002, 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20649 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:29.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337274+0000", 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337281+0000", 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6899999999999992e-06 2024-09-22T01:14:29.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337286+0000", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8040000000000001e-06 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382844+0000", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045558169000000003 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382943+0000", 2024-09-22T01:14:29.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8183999999999994e-05 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383645+0000", 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070229899999999998 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20650 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:29.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.342178076, 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046910647999999999, 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20650 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337294+0000", 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337296+0000", 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337303+0000", 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337308+0000", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5619999999999997e-06 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383674+0000", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046366387000000002 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383762+0000", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7583000000000005e-05 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384204+0000", 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000442402 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20651 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34215705899999999, 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124719000000002, 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20651 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337315+0000", 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337317+0000", 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337323+0000", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2700000000000001e-06 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337328+0000", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384232+0000", 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046903783999999997 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384346+0000", 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113687 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384439+0000", 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3484000000000001e-05 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20652 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34213685599999999, 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047279558999999999, 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20652 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337337+0000", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0370000000000001e-06 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337344+0000", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5780000000000004e-06 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337348+0000", 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6099999999999999e-06 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384469+0000", 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047120461000000002 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384517+0000", 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8649000000000001e-05 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384614+0000", 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6794999999999997e-05 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20653 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34211706200000003, 2024-09-22T01:14:29.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047782450999999997, 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20653 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337355+0000", 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337357+0000", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-06 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337363+0000", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337368+0000", 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3680000000000001e-06 2024-09-22T01:14:29.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384673+0000", 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047304572000000003 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384930+0000", 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000257574 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385137+0000", 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020650700000000001 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20654 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.341765967, 2024-09-22T01:14:29.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047643001999999997, 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20654 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4099999999999998e-07 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337710+0000", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-06 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337722+0000", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337729+0000", 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4479999999999997e-06 2024-09-22T01:14:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385171+0000", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047441496 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385252+0000", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1129000000000002e-05 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385349+0000", 2024-09-22T01:14:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6586e-05 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20655 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34172720400000001, 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047921126000000001, 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20655 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337745+0000", 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7299999999999996e-07 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337748+0000", 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7650000000000002e-06 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337755+0000", 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6129999999999998e-06 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337759+0000", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.309e-06 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385418+0000", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047659576000000002 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385552+0000", 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134179 2024-09-22T01:14:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385665+0000", 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011311099999999999 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20656 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [sparse-read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.341706644, 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048121733, 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20656 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337771+0000", 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2219999999999999e-06 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337776+0000", 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9429999999999999e-06 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337779+0000", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.836e-06 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385727+0000", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047947428 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385799+0000", 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2298999999999993e-05 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385887+0000", 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7614999999999994e-05 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20657 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.341687982, 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048317016999999997, 2024-09-22T01:14:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20657 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.72e-07 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337785+0000", 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2024-09-22T01:14:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337791+0000", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-06 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337804+0000", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2482e-05 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385927+0000", 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048123103 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386018+0000", 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1465000000000006e-05 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386101+0000", 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2224000000000004e-05 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20658 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34166268500000002, 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048416472000000002, 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20658 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5400000000000002e-07 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337811+0000", 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337815+0000", 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-22T01:14:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337818+0000", 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.734e-06 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386126+0000", 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048307911000000002 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386175+0000", 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8452000000000003e-05 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386225+0000", 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0717000000000002e-05 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20724 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34041989700000003, 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043947898999999999, 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20724 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339055+0000", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339063+0000", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3869999999999995e-06 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339068+0000", 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3890000000000004e-06 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382305+0000", 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043236827999999998 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382393+0000", 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8015000000000003e-05 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383000+0000", 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00060631200000000002 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20727 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34034999700000002, 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043983343000000001, 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20727 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.89e-07 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339125+0000", 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339132+0000", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8390000000000007e-06 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339146+0000", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3995000000000001e-05 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382923+0000", 2024-09-22T01:14:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043777099 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383006+0000", 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.297e-05 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383105+0000", 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8437999999999995e-05 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20728 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.340315162, 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044169813000000002, 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20728 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339157+0000", 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1399999999999998e-07 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339158+0000", 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.666e-06 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339164+0000", 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2889999999999997e-06 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339167+0000", 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8249999999999998e-06 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383147+0000", 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043979964000000003 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383250+0000", 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102815 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383326+0000", 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5939999999999995e-05 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20729 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7700480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.34029953600000001, 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383923999999998, 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20729 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.65e-07 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339179+0000", 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.387e-06 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339183+0000", 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9090000000000002e-06 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339185+0000", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7190000000000001e-06 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383369+0000", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044183275000000001 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383462+0000", 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3814000000000007e-05 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383556+0000", 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3554999999999995e-05 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20730 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.340282156, 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044529740999999998, 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20730 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339190+0000", 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7000000000000001e-07 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339198+0000", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8019999999999998e-06 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339205+0000", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7449999999999998e-06 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339210+0000", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9470000000000002e-06 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383584+0000", 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044374007 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383657+0000", 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2867000000000006e-05 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383719+0000", 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2102999999999998e-05 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:29.777 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:29.777 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:14:29.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:29.780 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:14:29.780 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:14:29.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:29.780 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:29.777+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:29.879 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:29.879+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:29.888 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone12 from 0xc8987d4 (overlap 0x113022b) to 0xc08184 2024-09-22T01:14:29.960 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:29.961 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:29.961 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:29.961 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:29.980 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:29.979+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:29.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:30.077 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-22T01:14:30.081 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:30.079+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:30.160 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:30.160 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:30.160 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:30.161 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:30.161 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:30.162 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:30.163 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:30.174 INFO:teuthology.orchestra.run.smithi096.stdout:1495 write 0x47261db thru 0x472d665 (0x748b bytes) 2024-09-22T01:14:30.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:30.182 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:30.181+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:30.183 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-22T01:14:29.732459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320861642,"num_objects":258,"num_object_clones":54,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":11,"num_read":26418,"num_read_kb":174579,"num_write":3767,"num_write_kb":73794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3042,"ondisk_log_size":3042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356012,"kb_used_data":106392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754618708,"statfs":{"total":773094113280,"available":772729556992,"internally_reserved":0,"allocated":108945408,"data_stored":137793626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":106521210,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":6784,"num_read_kb":63310,"num_write":888,"num_write_kb":36638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039805"},"pg_stats":[{"pgid":"3.1f","version":"45'47","reported_seq":573,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.804125+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:14:27.804125+0000","last_peered":"2024-09-22T01:14:27.804125+0000","last_clean":"2024-09-22T01:14:27.804125+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:14:27.804125+0000","last_undegraded":"2024-09-22T01:14:27.804125+0000","last_fullsized":"2024-09-22T01:14:27.804125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:09.134339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":7217213,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":479,"num_read_kb":9322,"num_write":45,"num_write_kb":793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"45'49","reported_seq":229,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.441369+0000","last_change":"2024-09-22T01:14:06.517323+0000","last_active":"2024-09-22T01:14:28.441369+0000","last_peered":"2024-09-22T01:14:28.441369+0000","last_clean":"2024-09-22T01:14:28.441369+0000","last_became_active":"2024-09-22T01:14:04.516518+0000","last_became_peered":"2024-09-22T01:14:04.516518+0000","last_unstale":"2024-09-22T01:14:28.441369+0000","last_undegraded":"2024-09-22T01:14:28.441369+0000","last_fullsized":"2024-09-22T01:14:28.441369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:31:40.429745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":11920179,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":3369,"num_write":51,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"41'42","reported_seq":492,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720319+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:14:27.720319+0000","last_peered":"2024-09-22T01:14:27.720319+0000","last_clean":"2024-09-22T01:14:27.720319+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:14:27.720319+0000","last_undegraded":"2024-09-22T01:14:27.720319+0000","last_fullsized":"2024-09-22T01:14:27.720319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:34.332393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":2392064,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":407,"num_read_kb":1485,"num_write":39,"num_write_kb":474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"45'60","reported_seq":712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.330133+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:14:28.330133+0000","last_peered":"2024-09-22T01:14:28.330133+0000","last_clean":"2024-09-22T01:14:28.330133+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:14:28.330133+0000","last_undegraded":"2024-09-22T01:14:28.330133+0000","last_fullsized":"2024-09-22T01:14:28.330133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:24.063242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":9068597,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":725,"num_read_kb":1328,"num_write":75,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'71","reported_seq":327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.319118+0000","last_change":"2024-09-22T01:14:06.510535+0000","last_active":"2024-09-22T01:14:28.319118+0000","last_peered":"2024-09-22T01:14:28.319118+0000","last_clean":"2024-09-22T01:14:28.319118+0000","last_became_active":"2024-09-22T01:14:04.502656+0000","last_became_peered":"2024-09-22T01:14:04.502656+0000","last_unstale":"2024-09-22T01:14:28.319118+0000","last_undegraded":"2024-09-22T01:14:28.319118+0000","last_fullsized":"2024-09-22T01:14:28.319118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:32.811743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":565285,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":473,"num_write":67,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"45'78","reported_seq":788,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.416953+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:14:28.416953+0000","last_peered":"2024-09-22T01:14:28.416953+0000","last_clean":"2024-09-22T01:14:28.416953+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:14:28.416953+0000","last_undegraded":"2024-09-22T01:14:28.416953+0000","last_fullsized":"2024-09-22T01:14:28.416953+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:36:39.041732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":15662170,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":806,"num_read_kb":6289,"num_write":102,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'55","reported_seq":743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713483+0000","last_change":"2024-09-22T01:14:06.509219+0000","last_active":"2024-09-22T01:14:27.713483+0000","last_peered":"2024-09-22T01:14:27.713483+0000","last_clean":"2024-09-22T01:14:27.713483+0000","last_became_active":"2024-09-22T01:14:04.511622+0000","last_became_peered":"2024-09-22T01:14:04.511622+0000","last_unstale":"2024-09-22T01:14:27.713483+0000","last_undegraded":"2024-09-22T01:14:27.713483+0000","last_fullsized":"2024-09-22T01:14:27.713483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:51.525132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":6525644,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1320,"num_read_kb":6420,"num_write":51,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"45'49","reported_seq":1026,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.274007+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:14:28.274007+0000","last_peered":"2024-09-22T01:14:28.274007+0000","last_clean":"2024-09-22T01:14:28.274007+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:14:28.274007+0000","last_undegraded":"2024-09-22T01:14:28.274007+0000","last_fullsized":"2024-09-22T01:14:28.274007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:18.199403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":8957246,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":933,"num_read_kb":9183,"num_write":46,"num_write_kb":604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"45'49","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.331387+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:14:28.331387+0000","last_peered":"2024-09-22T01:14:28.331387+0000","last_clean":"2024-09-22T01:14:28.331387+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:14:28.331387+0000","last_undegraded":"2024-09-22T01:14:28.331387+0000","last_fullsized":"2024-09-22T01:14:28.331387+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:37.693220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":11046650,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":97,"num_read_kb":2981,"num_write":43,"num_write_kb":1965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'128","reported_seq":1115,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.405615+0000","last_change":"2024-09-22T01:14:10.574662+0000","last_active":"2024-09-22T01:14:28.405615+0000","last_peered":"2024-09-22T01:14:28.405615+0000","last_clean":"2024-09-22T01:14:28.405615+0000","last_became_active":"2024-09-22T01:14:10.543526+0000","last_became_peered":"2024-09-22T01:14:10.543526+0000","last_unstale":"2024-09-22T01:14:28.405615+0000","last_undegraded":"2024-09-22T01:14:28.405615+0000","last_fullsized":"2024-09-22T01:14:28.405615+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:44:04.565285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":7785334,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":1211,"num_read_kb":8737,"num_write":152,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"44'129","reported_seq":480,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.529905+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:14:27.529905+0000","last_peered":"2024-09-22T01:14:27.529905+0000","last_clean":"2024-09-22T01:14:27.529905+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:14:27.529905+0000","last_undegraded":"2024-09-22T01:14:27.529905+0000","last_fullsized":"2024-09-22T01:14:27.529905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:51.591672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":4693276,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":738,"num_read_kb":5319,"num_write":198,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"45'103","reported_seq":621,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.490805+0000","last_change":"2024-09-22T01:14:10.560133+0000","last_active":"2024-09-22T01:14:28.490805+0000","last_peered":"2024-09-22T01:14:28.490805+0000","last_clean":"2024-09-22T01:14:28.490805+0000","last_became_active":"2024-09-22T01:14:10.539067+0000","last_became_peered":"2024-09-22T01:14:10.539067+0000","last_unstale":"2024-09-22T01:14:28.490805+0000","last_undegraded":"2024-09-22T01:14:28.490805+0000","last_fullsized":"2024-09-22T01:14:28.490805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:58.855692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":9148779,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":779,"num_read_kb":3509,"num_write":86,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720251+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:14:27.720251+0000","last_peered":"2024-09-22T01:14:27.720251+0000","last_clean":"2024-09-22T01:14:27.720251+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:14:27.720251+0000","last_undegraded":"2024-09-22T01:14:27.720251+0000","last_fullsized":"2024-09-22T01:14:27.720251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"45'181","reported_seq":1311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.427612+0000","last_change":"2024-09-22T01:14:06.510510+0000","last_active":"2024-09-22T01:14:28.427612+0000","last_peered":"2024-09-22T01:14:28.427612+0000","last_clean":"2024-09-22T01:14:28.427612+0000","last_became_active":"2024-09-22T01:14:04.523389+0000","last_became_peered":"2024-09-22T01:14:04.523389+0000","last_unstale":"2024-09-22T01:14:28.427612+0000","last_undegraded":"2024-09-22T01:14:28.427612+0000","last_fullsized":"2024-09-22T01:14:28.427612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:04.048774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":11337728,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2682,"num_read_kb":5911,"num_write":177,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729425+0000","last_change":"2024-09-22T01:14:17.618532+0000","last_active":"2024-09-22T01:14:27.729425+0000","last_peered":"2024-09-22T01:14:27.729425+0000","last_clean":"2024-09-22T01:14:27.729425+0000","last_became_active":"2024-09-22T01:14:17.618111+0000","last_became_peered":"2024-09-22T01:14:17.618111+0000","last_unstale":"2024-09-22T01:14:27.729425+0000","last_undegraded":"2024-09-22T01:14:27.729425+0000","last_fullsized":"2024-09-22T01:14:27.729425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:51.006648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'33","reported_seq":829,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713449+0000","last_change":"2024-09-22T01:14:06.510012+0000","last_active":"2024-09-22T01:14:27.713449+0000","last_peered":"2024-09-22T01:14:27.713449+0000","last_clean":"2024-09-22T01:14:27.713449+0000","last_became_active":"2024-09-22T01:14:04.529848+0000","last_became_peered":"2024-09-22T01:14:04.529848+0000","last_unstale":"2024-09-22T01:14:27.713449+0000","last_undegraded":"2024-09-22T01:14:27.713449+0000","last_fullsized":"2024-09-22T01:14:27.713449+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:36.415928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":4705062,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":746,"num_read_kb":5138,"num_write":40,"num_write_kb":795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720683+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:14:27.720683+0000","last_peered":"2024-09-22T01:14:27.720683+0000","last_clean":"2024-09-22T01:14:27.720683+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:14:27.720683+0000","last_undegraded":"2024-09-22T01:14:27.720683+0000","last_fullsized":"2024-09-22T01:14:27.720683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"45'42","reported_seq":438,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.429874+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:14:28.429874+0000","last_peered":"2024-09-22T01:14:28.429874+0000","last_clean":"2024-09-22T01:14:28.429874+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:14:28.429874+0000","last_undegraded":"2024-09-22T01:14:28.429874+0000","last_fullsized":"2024-09-22T01:14:28.429874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:47.504234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":3292018,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":751,"num_read_kb":3610,"num_write":50,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.734047+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:14:27.734047+0000","last_peered":"2024-09-22T01:14:27.734047+0000","last_clean":"2024-09-22T01:14:27.734047+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:14:27.734047+0000","last_undegraded":"2024-09-22T01:14:27.734047+0000","last_fullsized":"2024-09-22T01:14:27.734047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"44'34","reported_seq":248,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.724034+0000","last_change":"2024-09-22T01:14:06.509901+0000","last_active":"2024-09-22T01:14:27.724034+0000","last_peered":"2024-09-22T01:14:27.724034+0000","last_clean":"2024-09-22T01:14:27.724034+0000","last_became_active":"2024-09-22T01:14:04.528349+0000","last_became_peered":"2024-09-22T01:14:04.528349+0000","last_unstale":"2024-09-22T01:14:27.724034+0000","last_undegraded":"2024-09-22T01:14:27.724034+0000","last_fullsized":"2024-09-22T01:14:27.724034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:57:48.561765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":6594592,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":3374,"num_write":33,"num_write_kb":211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"42'46","reported_seq":379,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:26.726637+0000","last_change":"2024-09-22T01:14:06.521461+0000","last_active":"2024-09-22T01:14:26.726637+0000","last_peered":"2024-09-22T01:14:26.726637+0000","last_clean":"2024-09-22T01:14:26.726637+0000","last_became_active":"2024-09-22T01:14:04.507462+0000","last_became_peered":"2024-09-22T01:14:04.507462+0000","last_unstale":"2024-09-22T01:14:26.726637+0000","last_undegraded":"2024-09-22T01:14:26.726637+0000","last_fullsized":"2024-09-22T01:14:26.726637+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:11:52.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":6262115,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":303,"num_read_kb":4570,"num_write":54,"num_write_kb":3226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720074+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:14:27.720074+0000","last_peered":"2024-09-22T01:14:27.720074+0000","last_clean":"2024-09-22T01:14:27.720074+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:14:27.720074+0000","last_undegraded":"2024-09-22T01:14:27.720074+0000","last_fullsized":"2024-09-22T01:14:27.720074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.406203+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:14:28.406203+0000","last_peered":"2024-09-22T01:14:28.406203+0000","last_clean":"2024-09-22T01:14:28.406203+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:14:28.406203+0000","last_undegraded":"2024-09-22T01:14:28.406203+0000","last_fullsized":"2024-09-22T01:14:28.406203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:30.902835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":2490478,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":178,"num_read_kb":2865,"num_write":31,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.747519+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:14:27.747519+0000","last_peered":"2024-09-22T01:14:27.747519+0000","last_clean":"2024-09-22T01:14:27.747519+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:14:27.747519+0000","last_undegraded":"2024-09-22T01:14:27.747519+0000","last_fullsized":"2024-09-22T01:14:27.747519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'81","reported_seq":260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713084+0000","last_change":"2024-09-22T01:14:06.510830+0000","last_active":"2024-09-22T01:14:27.713084+0000","last_peered":"2024-09-22T01:14:27.713084+0000","last_clean":"2024-09-22T01:14:27.713084+0000","last_became_active":"2024-09-22T01:14:04.528610+0000","last_became_peered":"2024-09-22T01:14:04.528610+0000","last_unstale":"2024-09-22T01:14:27.713084+0000","last_undegraded":"2024-09-22T01:14:27.713084+0000","last_fullsized":"2024-09-22T01:14:27.713084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:28:49.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":434217,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":100,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.719769+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:14:27.719769+0000","last_peered":"2024-09-22T01:14:27.719769+0000","last_clean":"2024-09-22T01:14:27.719769+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:14:27.719769+0000","last_undegraded":"2024-09-22T01:14:27.719769+0000","last_fullsized":"2024-09-22T01:14:27.719769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"44'96","reported_seq":696,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713463+0000","last_change":"2024-09-22T01:14:06.509600+0000","last_active":"2024-09-22T01:14:27.713463+0000","last_peered":"2024-09-22T01:14:27.713463+0000","last_clean":"2024-09-22T01:14:27.713463+0000","last_became_active":"2024-09-22T01:14:04.512607+0000","last_became_peered":"2024-09-22T01:14:04.512607+0000","last_unstale":"2024-09-22T01:14:27.713463+0000","last_undegraded":"2024-09-22T01:14:27.713463+0000","last_fullsized":"2024-09-22T01:14:27.713463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:31:24.649161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":21020688,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":935,"num_read_kb":5136,"num_write":92,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720146+0000","last_change":"2024-09-22T01:14:10.551838+0000","last_active":"2024-09-22T01:14:27.720146+0000","last_peered":"2024-09-22T01:14:27.720146+0000","last_clean":"2024-09-22T01:14:27.720146+0000","last_became_active":"2024-09-22T01:14:10.539121+0000","last_became_peered":"2024-09-22T01:14:10.539121+0000","last_unstale":"2024-09-22T01:14:27.720146+0000","last_undegraded":"2024-09-22T01:14:27.720146+0000","last_fullsized":"2024-09-22T01:14:27.720146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:48.481715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729433+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:14:27.729433+0000","last_peered":"2024-09-22T01:14:27.729433+0000","last_clean":"2024-09-22T01:14:27.729433+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:14:27.729433+0000","last_undegraded":"2024-09-22T01:14:27.729433+0000","last_fullsized":"2024-09-22T01:14:27.729433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"45'101","reported_seq":597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.330514+0000","last_change":"2024-09-22T01:14:06.509555+0000","last_active":"2024-09-22T01:14:28.330514+0000","last_peered":"2024-09-22T01:14:28.330514+0000","last_clean":"2024-09-22T01:14:28.330514+0000","last_became_active":"2024-09-22T01:14:04.523352+0000","last_became_peered":"2024-09-22T01:14:04.523352+0000","last_unstale":"2024-09-22T01:14:28.330514+0000","last_undegraded":"2024-09-22T01:14:28.330514+0000","last_fullsized":"2024-09-22T01:14:28.330514+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:50.568196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":36865186,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":827,"num_read_kb":4387,"num_write":83,"num_write_kb":1346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"45'89","reported_seq":1078,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.472845+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:14:28.472845+0000","last_peered":"2024-09-22T01:14:28.472845+0000","last_clean":"2024-09-22T01:14:28.472845+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:14:28.472845+0000","last_undegraded":"2024-09-22T01:14:28.472845+0000","last_fullsized":"2024-09-22T01:14:28.472845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:05:28.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":42775764,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":986,"num_read_kb":21558,"num_write":86,"num_write_kb":8968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"45'94","reported_seq":865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.007075+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:14:28.007075+0000","last_peered":"2024-09-22T01:14:28.007075+0000","last_clean":"2024-09-22T01:14:28.007075+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:14:28.007075+0000","last_undegraded":"2024-09-22T01:14:28.007075+0000","last_fullsized":"2024-09-22T01:14:28.007075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:24.240784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":4587520,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1379,"num_read_kb":5086,"num_write":102,"num_write_kb":2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"44'223","reported_seq":1542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.747542+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:14:27.747542+0000","last_peered":"2024-09-22T01:14:27.747542+0000","last_clean":"2024-09-22T01:14:27.747542+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:14:27.747542+0000","last_undegraded":"2024-09-22T01:14:27.747542+0000","last_fullsized":"2024-09-22T01:14:27.747542+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:05.821326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":10831024,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":2231,"num_read_kb":5930,"num_write":372,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"44'91","reported_seq":1334,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.554761+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:14:27.554761+0000","last_peered":"2024-09-22T01:14:27.554761+0000","last_clean":"2024-09-22T01:14:27.554761+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:14:27.554761+0000","last_undegraded":"2024-09-22T01:14:27.554761+0000","last_fullsized":"2024-09-22T01:14:27.554761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:31.493804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":14244440,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1396,"num_read_kb":16322,"num_write":121,"num_write_kb":2225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"45'36","reported_seq":282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.063982+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:14:28.063982+0000","last_peered":"2024-09-22T01:14:28.063982+0000","last_clean":"2024-09-22T01:14:28.063982+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:14:28.063982+0000","last_undegraded":"2024-09-22T01:14:28.063982+0000","last_fullsized":"2024-09-22T01:14:28.063982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:23.072743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":6059408,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":4018,"num_write":40,"num_write_kb":1935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"42'121","reported_seq":596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.724375+0000","last_change":"2024-09-22T01:14:06.511183+0000","last_active":"2024-09-22T01:14:27.724375+0000","last_peered":"2024-09-22T01:14:27.724375+0000","last_clean":"2024-09-22T01:14:27.724375+0000","last_became_active":"2024-09-22T01:14:04.512742+0000","last_became_peered":"2024-09-22T01:14:04.512742+0000","last_unstale":"2024-09-22T01:14:27.724375+0000","last_undegraded":"2024-09-22T01:14:27.724375+0000","last_fullsized":"2024-09-22T01:14:27.724375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:01.283902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":5808924,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":827,"num_read_kb":4214,"num_write":174,"num_write_kb":1723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"40'86","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:26.726973+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:14:26.726973+0000","last_peered":"2024-09-22T01:14:26.726973+0000","last_clean":"2024-09-22T01:14:26.726973+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:14:26.726973+0000","last_undegraded":"2024-09-22T01:14:26.726973+0000","last_fullsized":"2024-09-22T01:14:26.726973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:56.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":3801139,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":304,"num_read_kb":3136,"num_write":108,"num_write_kb":1818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'31","reported_seq":202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729279+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:14:27.729279+0000","last_peered":"2024-09-22T01:14:27.729279+0000","last_clean":"2024-09-22T01:14:27.729279+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:14:27.729279+0000","last_undegraded":"2024-09-22T01:14:27.729279+0000","last_fullsized":"2024-09-22T01:14:27.729279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:37.914219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":1712128,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":110,"num_read_kb":1312,"num_write":31,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"45'177","reported_seq":684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.478039+0000","last_change":"2024-09-22T01:14:10.582207+0000","last_active":"2024-09-22T01:14:28.478039+0000","last_peered":"2024-09-22T01:14:28.478039+0000","last_clean":"2024-09-22T01:14:28.478039+0000","last_became_active":"2024-09-22T01:14:10.539572+0000","last_became_peered":"2024-09-22T01:14:10.539572+0000","last_unstale":"2024-09-22T01:14:28.478039+0000","last_undegraded":"2024-09-22T01:14:28.478039+0000","last_fullsized":"2024-09-22T01:14:28.478039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:20:06.980545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":12369863,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1200,"num_read_kb":7962,"num_write":223,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'294","reported_seq":435,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713146+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:14:27.713146+0000","last_peered":"2024-09-22T01:14:27.713146+0000","last_clean":"2024-09-22T01:14:27.713146+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:14:27.713146+0000","last_undegraded":"2024-09-22T01:14:27.713146+0000","last_fullsized":"2024-09-22T01:14:27.713146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:59:27.059358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":949148,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1358,"num_read_kb":1183,"num_write":473,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"44'174","reported_seq":720,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720043+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:14:27.720043+0000","last_peered":"2024-09-22T01:14:27.720043+0000","last_clean":"2024-09-22T01:14:27.720043+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:14:27.720043+0000","last_undegraded":"2024-09-22T01:14:27.720043+0000","last_fullsized":"2024-09-22T01:14:27.720043+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:23.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":29147376,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1101,"num_read_kb":9942,"num_write":261,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":320271255,"num_objects":254,"num_object_clones":54,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":11,"num_read":26302,"num_read_kb":174479,"num_write":3606,"num_write_kb":71376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":109019136,"data_stored":140257964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2922,"ondisk_log_size":2922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":8792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":9003008,"data_stored":9862323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":1392005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":16792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":17195008,"data_stored":19335801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46596,"kb_used_data":15400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325244,"statfs":{"total":96636764160,"available":96589049856,"internally_reserved":0,"allocated":15769600,"data_stored":16567183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":20148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":20631552,"data_stored":29873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45872,"kb_used_data":14684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325968,"statfs":{"total":96636764160,"available":96589791232,"internally_reserved":0,"allocated":15036416,"data_stored":20990275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50860,"kb_used_data":19628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320980,"statfs":{"total":96636764160,"available":96584683520,"internally_reserved":0,"allocated":20099072,"data_stored":24461177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40432,"kb_used_data":9248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331408,"statfs":{"total":96636764160,"available":96595361792,"internally_reserved":0,"allocated":9469952,"data_stored":15311697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14680064,"data_stored":20806223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18923520,"data_stored":23587707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14503936,"data_stored":20756113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20156416,"data_stored":29683559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15192064,"data_stored":16285885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":18374071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":1179070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8421376,"data_stored":9585336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:30.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:14:30.215 INFO:teuthology.orchestra.run.smithi096.stdout:1497 write 0x8700dc6 thru 0x8703c4e (0x2e89 bytes) 2024-09-22T01:14:30.226 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:30.226 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:30.232 INFO:teuthology.orchestra.run.smithi096.stdout:1502 trunc from 0xc8987d4 to 0x884eb79 2024-09-22T01:14:30.260 INFO:teuthology.orchestra.run.smithi096.stdout:1504 punch from 0x648d952 to 0x64946ee, (0x6d9c bytes) 2024-09-22T01:14:30.260 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:30.260 INFO:teuthology.orchestra.run.smithi096.stdout:1506 write 0x4d781 thru 0x5051d (0x2d9d bytes) 2024-09-22T01:14:30.282 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:30.281+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:30.286 INFO:teuthology.orchestra.run.smithi096.stdout:1507 clone 14 order 23 su 262144 sc 11 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:1738 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 131072~32768] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330852796999999, 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014237117000000001, 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1738 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006342+0000", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006357+0000", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5000999999999999e-05 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006366+0000", 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4619999999999996e-06 2024-09-22T01:14:30.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.006372+0000", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8379999999999998e-06 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013432+0000", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070601079999999998 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.013499+0000", 2024-09-22T01:14:30.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7531000000000001e-05 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.020579+0000", 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0070795700000000003 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2027 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3473408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330121678000001, 2024-09-22T01:14:30.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014158201, 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2027 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007073+0000", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007074+0000", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1399999999999997e-07 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007083+0000", 2024-09-22T01:14:30.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9889999999999998e-06 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007089+0000", 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5259999999999999e-06 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007094+0000", 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6530000000000002e-06 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021090+0000", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01399624 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021157+0000", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6538999999999994e-05 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021231+0000", 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4640000000000004e-05 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2028 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 3801088~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330094162000002, 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014289873, 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2028 2024-09-22T01:14:30.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007100+0000", 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007101+0000", 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007103+0000", 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007111+0000", 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007122+0000", 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0988000000000001e-05 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021258+0000", 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014136319 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021326+0000", 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7642000000000001e-05 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021390+0000", 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4645000000000002e-05 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330066087999999, 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:30.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:30.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330045419000001, 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:30.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.330015106000001, 2024-09-22T01:14:30.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:30.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:30.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:30.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329995852, 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:30.373 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329977750000001, 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:30.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:30.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329957026999999, 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:30.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:30.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329936537999998, 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:30.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:30.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:30.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:30.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.393 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329915096000001, 2024-09-22T01:14:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.395 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:30.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:30.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:30.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.403 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329884248999999, 2024-09-22T01:14:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:30.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:30.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:30.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329846722999999, 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:30.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329814594999998, 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:30.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:30.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.329791587999999, 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:30.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:30.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:30.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.290477755000001, 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:30.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:30.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:30.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.289190574999999, 2024-09-22T01:14:30.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:30.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:30.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:30.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.288846392, 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:30.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:30.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:30.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:15337 3.e 3:73ab052b:::rbd_data.1124645eb891.0000000000000038:c [sparse-read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7909868629999997, 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017128462000000001, 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15337 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546210+0000", 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:14:30.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546215+0000", 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3800000000000002e-06 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546219+0000", 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0620000000000002e-06 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554742+0000", 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085230519999999997 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554860+0000", 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117883 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.563336+0000", 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084759050000000006 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:21056 3.e 3:70da9618:::rbd_object_map.11247e2569dd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.43751549899999997, 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014372493, 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 21056 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.453 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899680+0000", 2024-09-22T01:14:30.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899683+0000", 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5669999999999999e-06 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899690+0000", 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.776e-06 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899696+0000", 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7790000000000001e-06 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911432+0000", 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011736394000000001 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911543+0000", 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110684 2024-09-22T01:14:30.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911906+0000", 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036337899999999999 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913199+0000", 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012926000000000001 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:30.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913966+0000", 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076740200000000004 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914016+0000", 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039e-05 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914052+0000", 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5404999999999997e-05 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.459 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:30.460 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:30.461 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:30.461+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:30.521 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:14:30.522 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:14:30.540 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-22T01:14:29.732459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":320861642,"num_objects":258,"num_object_clones":54,"num_object_copies":516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":258,"num_whiteouts":11,"num_read":26418,"num_read_kb":174579,"num_write":3767,"num_write_kb":73794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3042,"ondisk_log_size":3042,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356012,"kb_used_data":106392,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754618708,"statfs":{"total":773094113280,"available":772729556992,"internally_reserved":0,"allocated":108945408,"data_stored":137793626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":106521210,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":6784,"num_read_kb":63310,"num_write":888,"num_write_kb":36638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.039805"},"pg_stats":[{"pgid":"3.1f","version":"45'47","reported_seq":573,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.804125+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:14:27.804125+0000","last_peered":"2024-09-22T01:14:27.804125+0000","last_clean":"2024-09-22T01:14:27.804125+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:14:27.804125+0000","last_undegraded":"2024-09-22T01:14:27.804125+0000","last_fullsized":"2024-09-22T01:14:27.804125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:09.134339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":7217213,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":479,"num_read_kb":9322,"num_write":45,"num_write_kb":793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"45'49","reported_seq":229,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.441369+0000","last_change":"2024-09-22T01:14:06.517323+0000","last_active":"2024-09-22T01:14:28.441369+0000","last_peered":"2024-09-22T01:14:28.441369+0000","last_clean":"2024-09-22T01:14:28.441369+0000","last_became_active":"2024-09-22T01:14:04.516518+0000","last_became_peered":"2024-09-22T01:14:04.516518+0000","last_unstale":"2024-09-22T01:14:28.441369+0000","last_undegraded":"2024-09-22T01:14:28.441369+0000","last_fullsized":"2024-09-22T01:14:28.441369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:31:40.429745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":11920179,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":123,"num_read_kb":3369,"num_write":51,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"41'42","reported_seq":492,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720319+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:14:27.720319+0000","last_peered":"2024-09-22T01:14:27.720319+0000","last_clean":"2024-09-22T01:14:27.720319+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:14:27.720319+0000","last_undegraded":"2024-09-22T01:14:27.720319+0000","last_fullsized":"2024-09-22T01:14:27.720319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:34.332393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":2392064,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":407,"num_read_kb":1485,"num_write":39,"num_write_kb":474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"45'60","reported_seq":712,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.330133+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:14:28.330133+0000","last_peered":"2024-09-22T01:14:28.330133+0000","last_clean":"2024-09-22T01:14:28.330133+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:14:28.330133+0000","last_undegraded":"2024-09-22T01:14:28.330133+0000","last_fullsized":"2024-09-22T01:14:28.330133+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:24.063242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":9068597,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":725,"num_read_kb":1328,"num_write":75,"num_write_kb":6518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"45'71","reported_seq":327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.319118+0000","last_change":"2024-09-22T01:14:06.510535+0000","last_active":"2024-09-22T01:14:28.319118+0000","last_peered":"2024-09-22T01:14:28.319118+0000","last_clean":"2024-09-22T01:14:28.319118+0000","last_became_active":"2024-09-22T01:14:04.502656+0000","last_became_peered":"2024-09-22T01:14:04.502656+0000","last_unstale":"2024-09-22T01:14:28.319118+0000","last_undegraded":"2024-09-22T01:14:28.319118+0000","last_fullsized":"2024-09-22T01:14:28.319118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:15:32.811743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":565285,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":473,"num_write":67,"num_write_kb":267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"45'78","reported_seq":788,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.416953+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:14:28.416953+0000","last_peered":"2024-09-22T01:14:28.416953+0000","last_clean":"2024-09-22T01:14:28.416953+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:14:28.416953+0000","last_undegraded":"2024-09-22T01:14:28.416953+0000","last_fullsized":"2024-09-22T01:14:28.416953+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:36:39.041732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":15662170,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":806,"num_read_kb":6289,"num_write":102,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'55","reported_seq":743,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713483+0000","last_change":"2024-09-22T01:14:06.509219+0000","last_active":"2024-09-22T01:14:27.713483+0000","last_peered":"2024-09-22T01:14:27.713483+0000","last_clean":"2024-09-22T01:14:27.713483+0000","last_became_active":"2024-09-22T01:14:04.511622+0000","last_became_peered":"2024-09-22T01:14:04.511622+0000","last_unstale":"2024-09-22T01:14:27.713483+0000","last_undegraded":"2024-09-22T01:14:27.713483+0000","last_fullsized":"2024-09-22T01:14:27.713483+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:22:51.525132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":6525644,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1320,"num_read_kb":6420,"num_write":51,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"45'49","reported_seq":1026,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.274007+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:14:28.274007+0000","last_peered":"2024-09-22T01:14:28.274007+0000","last_clean":"2024-09-22T01:14:28.274007+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:14:28.274007+0000","last_undegraded":"2024-09-22T01:14:28.274007+0000","last_fullsized":"2024-09-22T01:14:28.274007+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:18.199403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":8957246,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":933,"num_read_kb":9183,"num_write":46,"num_write_kb":604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"45'49","reported_seq":192,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.331387+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:14:28.331387+0000","last_peered":"2024-09-22T01:14:28.331387+0000","last_clean":"2024-09-22T01:14:28.331387+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:14:28.331387+0000","last_undegraded":"2024-09-22T01:14:28.331387+0000","last_fullsized":"2024-09-22T01:14:28.331387+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:37.693220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":11046650,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":97,"num_read_kb":2981,"num_write":43,"num_write_kb":1965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'128","reported_seq":1115,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.405615+0000","last_change":"2024-09-22T01:14:10.574662+0000","last_active":"2024-09-22T01:14:28.405615+0000","last_peered":"2024-09-22T01:14:28.405615+0000","last_clean":"2024-09-22T01:14:28.405615+0000","last_became_active":"2024-09-22T01:14:10.543526+0000","last_became_peered":"2024-09-22T01:14:10.543526+0000","last_unstale":"2024-09-22T01:14:28.405615+0000","last_undegraded":"2024-09-22T01:14:28.405615+0000","last_fullsized":"2024-09-22T01:14:28.405615+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:44:04.565285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":7785334,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":1211,"num_read_kb":8737,"num_write":152,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"44'129","reported_seq":480,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.529905+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:14:27.529905+0000","last_peered":"2024-09-22T01:14:27.529905+0000","last_clean":"2024-09-22T01:14:27.529905+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:14:27.529905+0000","last_undegraded":"2024-09-22T01:14:27.529905+0000","last_fullsized":"2024-09-22T01:14:27.529905+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:51.591672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":4693276,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":738,"num_read_kb":5319,"num_write":198,"num_write_kb":1176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"45'103","reported_seq":621,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.490805+0000","last_change":"2024-09-22T01:14:10.560133+0000","last_active":"2024-09-22T01:14:28.490805+0000","last_peered":"2024-09-22T01:14:28.490805+0000","last_clean":"2024-09-22T01:14:28.490805+0000","last_became_active":"2024-09-22T01:14:10.539067+0000","last_became_peered":"2024-09-22T01:14:10.539067+0000","last_unstale":"2024-09-22T01:14:28.490805+0000","last_undegraded":"2024-09-22T01:14:28.490805+0000","last_fullsized":"2024-09-22T01:14:28.490805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:57:58.855692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":9148779,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":779,"num_read_kb":3509,"num_write":86,"num_write_kb":1613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720251+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:14:27.720251+0000","last_peered":"2024-09-22T01:14:27.720251+0000","last_clean":"2024-09-22T01:14:27.720251+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:14:27.720251+0000","last_undegraded":"2024-09-22T01:14:27.720251+0000","last_fullsized":"2024-09-22T01:14:27.720251+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"45'181","reported_seq":1311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.427612+0000","last_change":"2024-09-22T01:14:06.510510+0000","last_active":"2024-09-22T01:14:28.427612+0000","last_peered":"2024-09-22T01:14:28.427612+0000","last_clean":"2024-09-22T01:14:28.427612+0000","last_became_active":"2024-09-22T01:14:04.523389+0000","last_became_peered":"2024-09-22T01:14:04.523389+0000","last_unstale":"2024-09-22T01:14:28.427612+0000","last_undegraded":"2024-09-22T01:14:28.427612+0000","last_fullsized":"2024-09-22T01:14:28.427612+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:04.048774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":11337728,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":2682,"num_read_kb":5911,"num_write":177,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729425+0000","last_change":"2024-09-22T01:14:17.618532+0000","last_active":"2024-09-22T01:14:27.729425+0000","last_peered":"2024-09-22T01:14:27.729425+0000","last_clean":"2024-09-22T01:14:27.729425+0000","last_became_active":"2024-09-22T01:14:17.618111+0000","last_became_peered":"2024-09-22T01:14:17.618111+0000","last_unstale":"2024-09-22T01:14:27.729425+0000","last_undegraded":"2024-09-22T01:14:27.729425+0000","last_fullsized":"2024-09-22T01:14:27.729425+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:51.006648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'33","reported_seq":829,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713449+0000","last_change":"2024-09-22T01:14:06.510012+0000","last_active":"2024-09-22T01:14:27.713449+0000","last_peered":"2024-09-22T01:14:27.713449+0000","last_clean":"2024-09-22T01:14:27.713449+0000","last_became_active":"2024-09-22T01:14:04.529848+0000","last_became_peered":"2024-09-22T01:14:04.529848+0000","last_unstale":"2024-09-22T01:14:27.713449+0000","last_undegraded":"2024-09-22T01:14:27.713449+0000","last_fullsized":"2024-09-22T01:14:27.713449+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:36.415928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":4705062,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":746,"num_read_kb":5138,"num_write":40,"num_write_kb":795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720683+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:14:27.720683+0000","last_peered":"2024-09-22T01:14:27.720683+0000","last_clean":"2024-09-22T01:14:27.720683+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:14:27.720683+0000","last_undegraded":"2024-09-22T01:14:27.720683+0000","last_fullsized":"2024-09-22T01:14:27.720683+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"45'42","reported_seq":438,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.429874+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:14:28.429874+0000","last_peered":"2024-09-22T01:14:28.429874+0000","last_clean":"2024-09-22T01:14:28.429874+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:14:28.429874+0000","last_undegraded":"2024-09-22T01:14:28.429874+0000","last_fullsized":"2024-09-22T01:14:28.429874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:10:47.504234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":3292018,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":751,"num_read_kb":3610,"num_write":50,"num_write_kb":933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.734047+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:14:27.734047+0000","last_peered":"2024-09-22T01:14:27.734047+0000","last_clean":"2024-09-22T01:14:27.734047+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:14:27.734047+0000","last_undegraded":"2024-09-22T01:14:27.734047+0000","last_fullsized":"2024-09-22T01:14:27.734047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"44'34","reported_seq":248,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.724034+0000","last_change":"2024-09-22T01:14:06.509901+0000","last_active":"2024-09-22T01:14:27.724034+0000","last_peered":"2024-09-22T01:14:27.724034+0000","last_clean":"2024-09-22T01:14:27.724034+0000","last_became_active":"2024-09-22T01:14:04.528349+0000","last_became_peered":"2024-09-22T01:14:04.528349+0000","last_unstale":"2024-09-22T01:14:27.724034+0000","last_undegraded":"2024-09-22T01:14:27.724034+0000","last_fullsized":"2024-09-22T01:14:27.724034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:57:48.561765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":6594592,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":3374,"num_write":33,"num_write_kb":211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"42'46","reported_seq":379,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:26.726637+0000","last_change":"2024-09-22T01:14:06.521461+0000","last_active":"2024-09-22T01:14:26.726637+0000","last_peered":"2024-09-22T01:14:26.726637+0000","last_clean":"2024-09-22T01:14:26.726637+0000","last_became_active":"2024-09-22T01:14:04.507462+0000","last_became_peered":"2024-09-22T01:14:04.507462+0000","last_unstale":"2024-09-22T01:14:26.726637+0000","last_undegraded":"2024-09-22T01:14:26.726637+0000","last_fullsized":"2024-09-22T01:14:26.726637+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:11:52.320774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":6262115,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":303,"num_read_kb":4570,"num_write":54,"num_write_kb":3226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720074+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:14:27.720074+0000","last_peered":"2024-09-22T01:14:27.720074+0000","last_clean":"2024-09-22T01:14:27.720074+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:14:27.720074+0000","last_undegraded":"2024-09-22T01:14:27.720074+0000","last_fullsized":"2024-09-22T01:14:27.720074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"45'32","reported_seq":250,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.406203+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:14:28.406203+0000","last_peered":"2024-09-22T01:14:28.406203+0000","last_clean":"2024-09-22T01:14:28.406203+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:14:28.406203+0000","last_undegraded":"2024-09-22T01:14:28.406203+0000","last_fullsized":"2024-09-22T01:14:28.406203+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:14:30.902835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":2490478,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":178,"num_read_kb":2865,"num_write":31,"num_write_kb":240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.747519+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:14:27.747519+0000","last_peered":"2024-09-22T01:14:27.747519+0000","last_clean":"2024-09-22T01:14:27.747519+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:14:27.747519+0000","last_undegraded":"2024-09-22T01:14:27.747519+0000","last_fullsized":"2024-09-22T01:14:27.747519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'81","reported_seq":260,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713084+0000","last_change":"2024-09-22T01:14:06.510830+0000","last_active":"2024-09-22T01:14:27.713084+0000","last_peered":"2024-09-22T01:14:27.713084+0000","last_clean":"2024-09-22T01:14:27.713084+0000","last_became_active":"2024-09-22T01:14:04.528610+0000","last_became_peered":"2024-09-22T01:14:04.528610+0000","last_unstale":"2024-09-22T01:14:27.713084+0000","last_undegraded":"2024-09-22T01:14:27.713084+0000","last_fullsized":"2024-09-22T01:14:27.713084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:28:49.965037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":434217,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":504,"num_read_kb":410,"num_write":100,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":42,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.719769+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:14:27.719769+0000","last_peered":"2024-09-22T01:14:27.719769+0000","last_clean":"2024-09-22T01:14:27.719769+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:14:27.719769+0000","last_undegraded":"2024-09-22T01:14:27.719769+0000","last_fullsized":"2024-09-22T01:14:27.719769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"44'96","reported_seq":696,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713463+0000","last_change":"2024-09-22T01:14:06.509600+0000","last_active":"2024-09-22T01:14:27.713463+0000","last_peered":"2024-09-22T01:14:27.713463+0000","last_clean":"2024-09-22T01:14:27.713463+0000","last_became_active":"2024-09-22T01:14:04.512607+0000","last_became_peered":"2024-09-22T01:14:04.512607+0000","last_unstale":"2024-09-22T01:14:27.713463+0000","last_undegraded":"2024-09-22T01:14:27.713463+0000","last_fullsized":"2024-09-22T01:14:27.713463+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:31:24.649161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":21020688,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":935,"num_read_kb":5136,"num_write":92,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":46,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720146+0000","last_change":"2024-09-22T01:14:10.551838+0000","last_active":"2024-09-22T01:14:27.720146+0000","last_peered":"2024-09-22T01:14:27.720146+0000","last_clean":"2024-09-22T01:14:27.720146+0000","last_became_active":"2024-09-22T01:14:10.539121+0000","last_became_peered":"2024-09-22T01:14:10.539121+0000","last_unstale":"2024-09-22T01:14:27.720146+0000","last_undegraded":"2024-09-22T01:14:27.720146+0000","last_fullsized":"2024-09-22T01:14:27.720146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:05:48.481715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729433+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:14:27.729433+0000","last_peered":"2024-09-22T01:14:27.729433+0000","last_clean":"2024-09-22T01:14:27.729433+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:14:27.729433+0000","last_undegraded":"2024-09-22T01:14:27.729433+0000","last_fullsized":"2024-09-22T01:14:27.729433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"45'101","reported_seq":597,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.330514+0000","last_change":"2024-09-22T01:14:06.509555+0000","last_active":"2024-09-22T01:14:28.330514+0000","last_peered":"2024-09-22T01:14:28.330514+0000","last_clean":"2024-09-22T01:14:28.330514+0000","last_became_active":"2024-09-22T01:14:04.523352+0000","last_became_peered":"2024-09-22T01:14:04.523352+0000","last_unstale":"2024-09-22T01:14:28.330514+0000","last_undegraded":"2024-09-22T01:14:28.330514+0000","last_fullsized":"2024-09-22T01:14:28.330514+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:50.568196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":36865186,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":827,"num_read_kb":4387,"num_write":83,"num_write_kb":1346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"45'89","reported_seq":1078,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.472845+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:14:28.472845+0000","last_peered":"2024-09-22T01:14:28.472845+0000","last_clean":"2024-09-22T01:14:28.472845+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:14:28.472845+0000","last_undegraded":"2024-09-22T01:14:28.472845+0000","last_fullsized":"2024-09-22T01:14:28.472845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:05:28.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":42775764,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":986,"num_read_kb":21558,"num_write":86,"num_write_kb":8968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"45'94","reported_seq":865,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.007075+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:14:28.007075+0000","last_peered":"2024-09-22T01:14:28.007075+0000","last_clean":"2024-09-22T01:14:28.007075+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:14:28.007075+0000","last_undegraded":"2024-09-22T01:14:28.007075+0000","last_fullsized":"2024-09-22T01:14:28.007075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:24.240784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":4587520,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1379,"num_read_kb":5086,"num_write":102,"num_write_kb":2330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"44'223","reported_seq":1542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.747542+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:14:27.747542+0000","last_peered":"2024-09-22T01:14:27.747542+0000","last_clean":"2024-09-22T01:14:27.747542+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:14:27.747542+0000","last_undegraded":"2024-09-22T01:14:27.747542+0000","last_fullsized":"2024-09-22T01:14:27.747542+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:05.821326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":10831024,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":2231,"num_read_kb":5930,"num_write":372,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"44'91","reported_seq":1334,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.554761+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:14:27.554761+0000","last_peered":"2024-09-22T01:14:27.554761+0000","last_clean":"2024-09-22T01:14:27.554761+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:14:27.554761+0000","last_undegraded":"2024-09-22T01:14:27.554761+0000","last_fullsized":"2024-09-22T01:14:27.554761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:33:31.493804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":14244440,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1396,"num_read_kb":16322,"num_write":121,"num_write_kb":2225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"45'36","reported_seq":282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.063982+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:14:28.063982+0000","last_peered":"2024-09-22T01:14:28.063982+0000","last_clean":"2024-09-22T01:14:28.063982+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:14:28.063982+0000","last_undegraded":"2024-09-22T01:14:28.063982+0000","last_fullsized":"2024-09-22T01:14:28.063982+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:23.072743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":6059408,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":206,"num_read_kb":4018,"num_write":40,"num_write_kb":1935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"42'121","reported_seq":596,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.724375+0000","last_change":"2024-09-22T01:14:06.511183+0000","last_active":"2024-09-22T01:14:27.724375+0000","last_peered":"2024-09-22T01:14:27.724375+0000","last_clean":"2024-09-22T01:14:27.724375+0000","last_became_active":"2024-09-22T01:14:04.512742+0000","last_became_peered":"2024-09-22T01:14:04.512742+0000","last_unstale":"2024-09-22T01:14:27.724375+0000","last_undegraded":"2024-09-22T01:14:27.724375+0000","last_fullsized":"2024-09-22T01:14:27.724375+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:47:01.283902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":5808924,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":827,"num_read_kb":4214,"num_write":174,"num_write_kb":1723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"40'86","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-22T01:14:26.726973+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:14:26.726973+0000","last_peered":"2024-09-22T01:14:26.726973+0000","last_clean":"2024-09-22T01:14:26.726973+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:14:26.726973+0000","last_undegraded":"2024-09-22T01:14:26.726973+0000","last_fullsized":"2024-09-22T01:14:26.726973+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:53:56.154409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":3801139,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":304,"num_read_kb":3136,"num_write":108,"num_write_kb":1818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'31","reported_seq":202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.729279+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:14:27.729279+0000","last_peered":"2024-09-22T01:14:27.729279+0000","last_clean":"2024-09-22T01:14:27.729279+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:14:27.729279+0000","last_undegraded":"2024-09-22T01:14:27.729279+0000","last_fullsized":"2024-09-22T01:14:27.729279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:37.914219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":1712128,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":110,"num_read_kb":1312,"num_write":31,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"45'177","reported_seq":684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:28.478039+0000","last_change":"2024-09-22T01:14:10.582207+0000","last_active":"2024-09-22T01:14:28.478039+0000","last_peered":"2024-09-22T01:14:28.478039+0000","last_clean":"2024-09-22T01:14:28.478039+0000","last_became_active":"2024-09-22T01:14:10.539572+0000","last_became_peered":"2024-09-22T01:14:10.539572+0000","last_unstale":"2024-09-22T01:14:28.478039+0000","last_undegraded":"2024-09-22T01:14:28.478039+0000","last_fullsized":"2024-09-22T01:14:28.478039+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:20:06.980545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":12369863,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1200,"num_read_kb":7962,"num_write":223,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'294","reported_seq":435,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.713146+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:14:27.713146+0000","last_peered":"2024-09-22T01:14:27.713146+0000","last_clean":"2024-09-22T01:14:27.713146+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:14:27.713146+0000","last_undegraded":"2024-09-22T01:14:27.713146+0000","last_fullsized":"2024-09-22T01:14:27.713146+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:59:27.059358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":949148,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1358,"num_read_kb":1183,"num_write":473,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"44'174","reported_seq":720,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-22T01:14:27.720043+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:14:27.720043+0000","last_peered":"2024-09-22T01:14:27.720043+0000","last_clean":"2024-09-22T01:14:27.720043+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:14:27.720043+0000","last_undegraded":"2024-09-22T01:14:27.720043+0000","last_fullsized":"2024-09-22T01:14:27.720043+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:01:23.493185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":29147376,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1101,"num_read_kb":9942,"num_write":261,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":320271255,"num_objects":254,"num_object_clones":54,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":11,"num_read":26302,"num_read_kb":174479,"num_write":3606,"num_write_kb":71376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":109019136,"data_stored":140257964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2922,"ondisk_log_size":2922,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39976,"kb_used_data":8792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331864,"statfs":{"total":96636764160,"available":96595828736,"internally_reserved":0,"allocated":9003008,"data_stored":9862323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32892,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338948,"statfs":{"total":96636764160,"available":96603082752,"internally_reserved":0,"allocated":1740800,"data_stored":1392005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":16792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":17195008,"data_stored":19335801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46596,"kb_used_data":15400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325244,"statfs":{"total":96636764160,"available":96589049856,"internally_reserved":0,"allocated":15769600,"data_stored":16567183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509453,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":20148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":20631552,"data_stored":29873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45872,"kb_used_data":14684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325968,"statfs":{"total":96636764160,"available":96589791232,"internally_reserved":0,"allocated":15036416,"data_stored":20990275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50860,"kb_used_data":19628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320980,"statfs":{"total":96636764160,"available":96584683520,"internally_reserved":0,"allocated":20099072,"data_stored":24461177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40432,"kb_used_data":9248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331408,"statfs":{"total":96636764160,"available":96595361792,"internally_reserved":0,"allocated":9469952,"data_stored":15311697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14680064,"data_stored":20806223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":18923520,"data_stored":23587707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14503936,"data_stored":20756113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20156416,"data_stored":29683559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15192064,"data_stored":16285885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15908864,"data_stored":18374071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1232896,"data_stored":1179070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8421376,"data_stored":9585336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:14:30.542 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:14:30.542 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:30.542 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-22T01:14:30.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-22T01:14:30.561 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:30.561+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:30.618 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:30.618 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:30.618 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:30.619 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:30.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:30.662 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:30.661+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:30.733 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:14:30.762 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:30.761+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:30.806 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:30.806 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:30.806 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:30.806 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:30.807 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:30.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:30.867 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:30.867+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:30.870 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone13 from 0x884eb79 (overlap 0x884eb79) to 0x28e46a4 2024-09-22T01:14:30.922 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-22T01:14:30.919+0000 7f33f656f640 -1 mon.a@0(leader).osd e48 definitely_dead 0 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.930040155, 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:14:30.950 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:14:30.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:14:30.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2200 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7110656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925474606999998, 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046602256000000002, 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2200 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024422+0000", 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024429+0000", 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024434+0000", 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070876+0000", 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046442215000000002 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070950+0000", 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3189999999999996e-05 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071022+0000", 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2273000000000004e-05 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.961 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2201 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925452112999999, 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046753765000000003, 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2201 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:14:30.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024445+0000", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3489999999999999e-06 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024451+0000", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4470000000000003e-06 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024457+0000", 2024-09-22T01:14:30.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071061+0000", 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046603415000000002 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071123+0000", 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2509999999999996e-05 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071196+0000", 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2719000000000006e-05 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2202 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925429118, 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046876332, 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2202 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:30.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024468+0000", 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024474+0000", 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024480+0000", 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071230+0000", 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046749717000000003 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071298+0000", 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8266e-05 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071341+0000", 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3350000000000003e-05 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2203 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.970 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925406894999998, 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046984537999999999, 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2203 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:30.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024488+0000", 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024490+0000", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024497+0000", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9970000000000001e-06 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024503+0000", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.31e-06 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071374+0000", 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046871082000000001 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071423+0000", 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9172999999999998e-05 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071472+0000", 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8973e-05 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2211 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.92538429, 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124055999999998, 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2211 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3399999999999999e-07 2024-09-22T01:14:30.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024513+0000", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024520+0000", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7410000000000003e-06 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024533+0000", 2024-09-22T01:14:30.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2867999999999999e-05 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071509+0000", 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046976047 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071569+0000", 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0482999999999997e-05 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071634+0000", 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4834999999999998e-05 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2212 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925353457, 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047215269999999997, 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2212 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:30.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3e-07 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024544+0000", 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0690000000000002e-06 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024551+0000", 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.686e-06 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024557+0000", 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.806e-06 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071661+0000", 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047104241999999998 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071703+0000", 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071756+0000", 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3171999999999998e-05 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2213 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.983 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925318361999999, 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047299709000000002, 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2213 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:30.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024593+0000", 2024-09-22T01:14:30.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6223e-05 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024600+0000", 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024607+0000", 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080000000000003e-06 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071783+0000", 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047175863999999998 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071822+0000", 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071875+0000", 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3801999999999998e-05 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2214 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.925278485, 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047393246999999999, 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2214 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026492+0000", 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018767059999999999 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026500+0000", 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026513+0000", 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3422e-05 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026523+0000", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071904+0000", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045380993000000001 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071942+0000", 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572999999999998e-05 2024-09-22T01:14:30.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.072009+0000", 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6797000000000004e-05 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2405 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 4915200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920755583999998, 2024-09-22T01:14:30.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04671264, 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2405 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029140+0000", 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1209999999999999e-06 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029143+0000", 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2049999999999998e-06 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029146+0000", 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7800000000000001e-06 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075719+0000", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046573497999999998 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075787+0000", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7135e-05 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075851+0000", 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4660999999999996e-05 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:30.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2406 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920744640999999, 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046865920999999998, 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2406 2024-09-22T01:14:30.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:30.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.22e-07 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029151+0000", 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:30.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029154+0000", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.439e-06 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029158+0000", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075881+0000", 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046723187999999999 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075951+0000", 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9565000000000002e-05 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076015+0000", 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4931000000000006e-05 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.001 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2407 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920732426000001, 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047019894, 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2407 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3900000000000001e-07 2024-09-22T01:14:31.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029163+0000", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2190000000000001e-06 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029167+0000", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3390000000000001e-06 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029169+0000", 2024-09-22T01:14:31.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9249999999999999e-06 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076045+0000", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046875173999999999 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076114+0000", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.902e-05 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076182+0000", 2024-09-22T01:14:31.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7978000000000005e-05 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2408 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920720924000001, 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047176774999999997, 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2408 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.35e-07 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029175+0000", 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029178+0000", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2650000000000001e-06 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029181+0000", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6860000000000002e-06 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076216+0000", 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035615000000003 2024-09-22T01:14:31.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076286+0000", 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9680999999999999e-05 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076350+0000", 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4172999999999997e-05 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2409 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920709928000001, 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047331204000000002, 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2409 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-07 2024-09-22T01:14:31.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029186+0000", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029190+0000", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1169999999999997e-06 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029194+0000", 2024-09-22T01:14:31.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5310000000000003e-06 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076377+0000", 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047182737000000002 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076447+0000", 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0411000000000001e-05 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076515+0000", 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7960999999999995e-05 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2410 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920677858000001, 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047479322999999997, 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2410 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029217+0000", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029219+0000", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.339e-06 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029226+0000", 2024-09-22T01:14:31.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6429999999999998e-06 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029231+0000", 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.152e-06 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076544+0000", 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047312694000000002 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076630+0000", 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5753000000000004e-05 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076696+0000", 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6087999999999995e-05 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2411 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.019 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920648536000002, 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047606067000000002, 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2411 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029249+0000", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4739999999999999e-06 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029256+0000", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999996e-06 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029260+0000", 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2024-09-22T01:14:31.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076724+0000", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047463678000000002 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076786+0000", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000005e-05 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076852+0000", 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5759999999999994e-05 2024-09-22T01:14:31.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2412 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.9206264, 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047679747000000001, 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2412 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029270+0000", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0540000000000002e-06 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029277+0000", 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-09-22T01:14:31.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029282+0000", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6890000000000003e-06 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076869+0000", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047587741000000003 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076909+0000", 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9504e-05 2024-09-22T01:14:31.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076947+0000", 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8578999999999998e-05 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2413 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920604302000001, 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047815037999999997, 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2413 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029292+0000", 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029299+0000", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029304+0000", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076974+0000", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047670055000000003 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077036+0000", 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2019000000000004e-05 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077105+0000", 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9127000000000004e-05 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2414 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.920583571999998, 2024-09-22T01:14:31.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048254547000000002, 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2414 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029313+0000", 2024-09-22T01:14:31.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.075e-06 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029320+0000", 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029325+0000", 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077133+0000", 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047808311999999999 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077197+0000", 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3912999999999996e-05 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077565+0000", 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830000000000001 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3931 3.11 3:8f505f18:::rbd_data.1124ef2748ce.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.834715376999998, 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048441529999999997, 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3931 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:31.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115182+0000", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115189+0000", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115194+0000", 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:31.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160040+0000", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044845812999999998 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160142+0000", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010177499999999999 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160287+0000", 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014540899999999999 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163457+0000", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031691570000000001 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163508+0000", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1944000000000003e-05 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163581+0000", 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2123e-05 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:31.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163620+0000", 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709000000000002e-05 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:31.042 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:31.043 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:31.044 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:31.043+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:31.127 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-22T01:14:31.144 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:31.145+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:31.178 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:31.178 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:31.178 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:31.194 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:31.197 INFO:teuthology.orchestra.run.smithi096.stdout:1508 write 0xd0c790c thru 0xd0d62e0 (0xe9d5 bytes) 2024-09-22T01:14:31.204 INFO:teuthology.orchestra.run.smithi096.stdout:1509 write 0x6ce3abe thru 0x6cf2d69 (0xf2ac bytes) 2024-09-22T01:14:31.237 INFO:teuthology.orchestra.run.smithi096.stdout:1510 trunc from 0xd0d62e1 to 0x2a30188 2024-09-22T01:14:31.284 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:31.285+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:31.332 INFO:teuthology.orchestra.run.smithi096.stdout:1512 punch from 0x273f72e to 0x274016e, (0xa40 bytes) 2024-09-22T01:14:31.345 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:31.343+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:31.350 INFO:teuthology.orchestra.run.smithi096.stdout:1513 writesame 0x5bfc202 thru 0x5c04697 data_size 0x3(0x8496 bytes) 2024-09-22T01:14:31.351 INFO:teuthology.orchestra.run.smithi096.stdout:1514 punch from 0x1b49195 to 0x1b58749, (0xf5b4 bytes) 2024-09-22T01:14:31.363 INFO:teuthology.orchestra.run.smithi096.stdout:1515 trunc from 0x5c04698 to 0x2fe3ec 2024-09-22T01:14:31.381 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:31.381 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:31.382 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:31.382 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:31.382 INFO:teuthology.orchestra.run.smithi096.stdout:1517 punch from 0x2bf487 to 0x2c16d2, (0x224b bytes) 2024-09-22T01:14:31.395 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:31.396 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:31.405 INFO:teuthology.orchestra.run.smithi096.stdout:1518 write 0x5a2b31f thru 0x5a2cc37 (0x1919 bytes) 2024-09-22T01:14:31.434 INFO:teuthology.orchestra.run.smithi096.stdout:1519 write 0xed6385e thru 0xed712e2 (0xda85 bytes) 2024-09-22T01:14:31.440 INFO:teuthology.orchestra.run.smithi096.stdout:1520 write 0xa8c027a thru 0xa8c7d4f (0x7ad6 bytes) 2024-09-22T01:14:31.445 INFO:teuthology.orchestra.run.smithi096.stdout:1523 read 0xe76ac0 thru 0xe7791e (0xe5f bytes) 2024-09-22T01:14:31.446 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:31.443+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:31.446 INFO:teuthology.orchestra.run.smithi096.stdout:1525 write 0x37dea93 thru 0x37e10bd (0x262b bytes) 2024-09-22T01:14:31.448 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.448 INFO:teuthology.orchestra.run.smithi096.stdout:1528 read 0x5ff8223 thru 0x6001feb (0x9dc9 bytes) 2024-09-22T01:14:31.448 INFO:teuthology.orchestra.run.smithi096.stdout:1530 read 0x744b70f thru 0x744c904 (0x11f6 bytes) 2024-09-22T01:14:31.449 INFO:teuthology.orchestra.run.smithi096.stdout:1531 read 0xae0ed0a thru 0xae12f31 (0x4228 bytes) 2024-09-22T01:14:31.449 INFO:teuthology.orchestra.run.smithi096.stdout:1532 writesame 0x907f6d6 thru 0x908c0a9 data_size 0x2(0xc9d4 bytes) 2024-09-22T01:14:31.481 INFO:teuthology.orchestra.run.smithi096.stdout:1533 writesame 0xbb8a9d9 thru 0xbb90c89 data_size 0x1f(0x62b1 bytes) 2024-09-22T01:14:31.489 INFO:teuthology.orchestra.run.smithi096.stdout:1534 write 0x64cc584 thru 0x64d379b (0x7218 bytes) 2024-09-22T01:14:31.503 INFO:teuthology.orchestra.run.smithi096.stdout:1535 trunc from 0xed712e3 to 0xaff067e 2024-09-22T01:14:31.520 INFO:teuthology.orchestra.run.smithi096.stdout:1536 read 0xa157e0f thru 0xa15af02 (0x30f4 bytes) 2024-09-22T01:14:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:1538 punch from 0x68033c4 to 0x6803e9d, (0xad9 bytes) 2024-09-22T01:14:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:1541 read 0x4f8f9c8 thru 0x4f99183 (0x97bc bytes) 2024-09-22T01:14:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:1543 punch from 0xa38696 to 0xa417b4, (0x911e bytes) 2024-09-22T01:14:31.534 INFO:teuthology.orchestra.run.smithi096.stdout:1545 writesame 0xe59cd96 thru 0xe5a4ea2 data_size 0x1(0x810d bytes) 2024-09-22T01:14:31.534 INFO:teuthology.orchestra.run.smithi096.stdout:1546 write 0xd134b38 thru 0xd137a2b (0x2ef4 bytes) 2024-09-22T01:14:31.547 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:31.547 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:31.547 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:31.547+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:31.558 INFO:teuthology.orchestra.run.smithi096.stdout:1548 trunc from 0xe5a4ea3 to 0x1ec0032 2024-09-22T01:14:31.565 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:31.565 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:31.574 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:31.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2216407399999998, 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:31.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:31.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2215745739999999, 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.221514537, 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2214794750000002, 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.591 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2214492049999999, 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:31.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:31.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2214196319999999, 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2213655409999999, 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.600 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2213096970000001, 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:31.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2212518069999998, 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:31.609 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.613 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2211900739999999, 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.614 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2210946109999998, 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.618 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:31.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:31.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.622 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.221032788, 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.623 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:31.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:31.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2209416480000002, 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.627 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:31.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:31.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:31.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2208797420000002, 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:31.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:31.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2208192549999999, 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:31.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:31.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2207617850000001, 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:31.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2207045050000001, 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.220644981, 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:31.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2205920969999999, 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:31.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:31.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:31.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 2.2114795350000001, 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:31.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:31.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:31.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:31.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:31.663 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:31.664 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:31.664 INFO:teuthology.orchestra.run.smithi096.stdout:1550 trunc from 0x1ec0032 to 0x32fdf85 2024-09-22T01:14:31.665 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:31.665 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:31.665 INFO:teuthology.orchestra.run.smithi096.stdout:1551 trunc from 0x32fdf85 to 0x1be776 2024-09-22T01:14:31.666 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:31.666 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:31.666 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:31.667 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:31.667 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:31.667 INFO:teuthology.orchestra.run.smithi096.stdout:1552 write 0x4f3c698 thru 0x4f42c96 (0x65ff bytes) 2024-09-22T01:14:31.667 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:31.668 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:1555 trunc from 0x4f42c97 to 0xe6641d4 2024-09-22T01:14:31.668 INFO:teuthology.orchestra.run.smithi096.stdout:1556 read 0x36e7112 thru 0x36f4244 (0xd133 bytes) 2024-09-22T01:14:31.669 INFO:teuthology.orchestra.run.smithi096.stdout:1558 trunc from 0xe6641d4 to 0xad5df52 2024-09-22T01:14:31.669 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:31.663+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:31.669 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:31.669 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:31.672 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.672 INFO:teuthology.orchestra.run.smithi096.stdout:1560 write 0x757e1 thru 0x814e1 (0xbd01 bytes) 2024-09-22T01:14:31.675 INFO:teuthology.orchestra.run.smithi096.stdout:1561 write 0x22095e4 thru 0x221616f (0xcb8c bytes) 2024-09-22T01:14:31.678 INFO:teuthology.orchestra.run.smithi096.stdout:1564 read 0xa5d263f thru 0xa5d5dc9 (0x378b bytes) 2024-09-22T01:14:31.678 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.678 INFO:teuthology.orchestra.run.smithi096.stdout:1566 read 0x35720e3 thru 0x35812b2 (0xf1d0 bytes) 2024-09-22T01:14:31.679 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:31.679 INFO:teuthology.orchestra.run.smithi096.stdout:1571 write 0x18d1ec6 thru 0x18d8f99 (0x70d4 bytes) 2024-09-22T01:14:31.681 INFO:teuthology.orchestra.run.smithi096.stdout:1572 clone 15 order 24 su 1048576 sc 8 2024-09-22T01:14:31.687 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:31.688 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:31.754 INFO:teuthology.orchestra.run.smithi096.stderr:marked out osd.2. 2024-09-22T01:14:31.765 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:31.763+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:31.798 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:31.832 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:31.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:31.866 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:31.863+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:31.956 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:31.957 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:31.967 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:31.967+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:31.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:32.068 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:32.069+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:32.093 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.093 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.107 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:32.196 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:32.197+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:32.271 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:32.267+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:32.276 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.276 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.290 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:32.376 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:32.377+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:32.457 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.457 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:32.512 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:32.513+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:32.574 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:32.573+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:32.631 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.631 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:32.688 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:32.689+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:32.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:14:32.807 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:32.807+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:32.826 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.826 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:32.888 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:32.889+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:32.971 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:32.972 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:32.977 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:32.977+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:32.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:33.077 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:33.077+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:33.150 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:33.150 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:33.165 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:33.211 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:33.211+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:33.279 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:33.277+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:33.290 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:33.290 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:33.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:33.380 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:33.381+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:33.471 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:33.471 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:33.481 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:33.479+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:33.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:33.582 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:33.579+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:33.651 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:33.651 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:33.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:33.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:33.711 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:33.711+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:33.765 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:14:33.784 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:33.785+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:33.866 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone14 from 0xad5df52 (overlap 0x1be776) to 0x0 2024-09-22T01:14:33.869 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:33.869 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:33.885 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:33.883+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:33.886 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:33.889 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:33.889 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:33.909 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:33.910 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:33.933 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:33.934 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:33.954 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:33.954 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:33.976 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:33.976 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:33.986 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:33.983+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:33.999 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:34.000 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:34.020 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:34.020 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:34.020 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:34.023 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:34.023 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:34.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:34.048 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:34.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:34.077 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-22T01:14:34.086 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:34.083+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:34.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.186801309000003, 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:34.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:14:34.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:14:34.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20645 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 3670016~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.834206891, 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044103126999999999, 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20645 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337062+0000", 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337063+0000", 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4600000000000005e-07 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337066+0000", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9050000000000001e-06 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337077+0000", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0693000000000001e-05 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337083+0000", 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2559999999999999e-06 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.380731+0000", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043648570999999997 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.380790+0000", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8174999999999998e-05 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381166+0000", 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000375981 2024-09-22T01:14:34.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20646 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8341782020000004, 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044647025, 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20646 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337091+0000", 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337092+0000", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1500000000000004e-07 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337094+0000", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2230000000000001e-06 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337101+0000", 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337105+0000", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8389999999999997e-06 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381224+0000", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044118460999999998 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381315+0000", 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1325999999999999e-05 2024-09-22T01:14:34.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381738+0000", 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042295400000000002 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20647 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.834157298, 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045060006, 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20647 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337112+0000", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9099999999999999e-07 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337115+0000", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2500000000000001e-06 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337123+0000", 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1790000000000008e-06 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337128+0000", 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7639999999999996e-06 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381789+0000", 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044661197 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.381855+0000", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6688999999999998e-05 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382172+0000", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031653600000000002 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20648 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:34.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8341342540000003, 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045676618000000002, 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20648 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:34.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337135+0000", 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337136+0000", 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337145+0000", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7540000000000001e-06 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337238+0000", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2487999999999999e-05 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337258+0000", 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0245e-05 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382230+0000", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044971934999999998 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382345+0000", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001155 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382812+0000", 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046620400000000001 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20649 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [sparse-read 5013504~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8339995089999999, 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046375128000000002, 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20649 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337270+0000", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337274+0000", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3859999999999999e-06 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337281+0000", 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6899999999999992e-06 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337286+0000", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8040000000000001e-06 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382844+0000", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045558169000000003 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382943+0000", 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8183999999999994e-05 2024-09-22T01:14:34.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383645+0000", 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00070229899999999998 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20650 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8339758709999998, 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046910647999999999, 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20650 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337293+0000", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337294+0000", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337296+0000", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1169999999999998e-06 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337303+0000", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1409999999999999e-06 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337308+0000", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5619999999999997e-06 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383674+0000", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046366387000000002 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383762+0000", 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7583000000000005e-05 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384204+0000", 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000442402 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20651 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:34.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8339548539999999, 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124719000000002, 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20651 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337314+0000", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337315+0000", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337317+0000", 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-22T01:14:34.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337323+0000", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2700000000000001e-06 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337328+0000", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384232+0000", 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046903783999999997 2024-09-22T01:14:34.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384346+0000", 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113687 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384439+0000", 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3484000000000001e-05 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20652 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8339346509999999, 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047279558999999999, 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20652 2024-09-22T01:14:34.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337335+0000", 2024-09-22T01:14:34.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2899999999999999e-07 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337337+0000", 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0370000000000001e-06 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337344+0000", 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5780000000000004e-06 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337348+0000", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6099999999999999e-06 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384469+0000", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047120461000000002 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384517+0000", 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8649000000000001e-05 2024-09-22T01:14:34.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384614+0000", 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6794999999999997e-05 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20653 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8339148569999999, 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047782450999999997, 2024-09-22T01:14:34.214 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20653 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337354+0000", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337355+0000", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337357+0000", 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8479999999999999e-06 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337363+0000", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337368+0000", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3680000000000001e-06 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384673+0000", 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047304572000000003 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.384930+0000", 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000257574 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385137+0000", 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00020650700000000001 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20654 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8335637619999998, 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047643001999999997, 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20654 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337706+0000", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4099999999999998e-07 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337710+0000", 2024-09-22T01:14:34.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-06 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337722+0000", 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337729+0000", 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4479999999999997e-06 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385171+0000", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047441496 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385252+0000", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1129000000000002e-05 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385349+0000", 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6586e-05 2024-09-22T01:14:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20655 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8335249989999998, 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047921126000000001, 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20655 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337744+0000", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337745+0000", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7299999999999996e-07 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337748+0000", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7650000000000002e-06 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337755+0000", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6129999999999998e-06 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337759+0000", 2024-09-22T01:14:34.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.309e-06 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385418+0000", 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047659576000000002 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385552+0000", 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134179 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385665+0000", 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011311099999999999 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20656 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [sparse-read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8335044390000004, 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048121733, 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20656 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:34.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337765+0000", 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337771+0000", 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2219999999999999e-06 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337776+0000", 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9429999999999999e-06 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337779+0000", 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.836e-06 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385727+0000", 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047947428 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385799+0000", 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2298999999999993e-05 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385887+0000", 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7614999999999994e-05 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20657 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8334857769999999, 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048317016999999997, 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20657 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337784+0000", 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.72e-07 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337785+0000", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6109999999999999e-06 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337791+0000", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8599999999999998e-06 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337804+0000", 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2482e-05 2024-09-22T01:14:34.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.385927+0000", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048123103 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386018+0000", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1465000000000006e-05 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386101+0000", 2024-09-22T01:14:34.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2224000000000004e-05 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20658 3.1a 3:5d213ce4:::rbd_data.11249227f48c.0000000000000001:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8334604800000003, 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048416472000000002, 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20658 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337809+0000", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5400000000000002e-07 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337811+0000", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6589999999999999e-06 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337815+0000", 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.337818+0000", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.734e-06 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386126+0000", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048307911000000002 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386175+0000", 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8452000000000003e-05 2024-09-22T01:14:34.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.386225+0000", 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0717000000000002e-05 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20724 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 6029312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8322176920000004, 2024-09-22T01:14:34.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043947898999999999, 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20724 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:34.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339052+0000", 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1999999999999999e-07 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339055+0000", 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339063+0000", 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3869999999999995e-06 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339068+0000", 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3890000000000004e-06 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382305+0000", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043236827999999998 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382393+0000", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8015000000000003e-05 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383000+0000", 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00060631200000000002 2024-09-22T01:14:34.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20727 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8321477919999998, 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043983343000000001, 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20727 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339122+0000", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.89e-07 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339125+0000", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5129999999999999e-06 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339132+0000", 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8390000000000007e-06 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339146+0000", 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3995000000000001e-05 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.382923+0000", 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043777099 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383006+0000", 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.297e-05 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383105+0000", 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8437999999999995e-05 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20728 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8321129569999997, 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044169813000000002, 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20728 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339156+0000", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339157+0000", 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1399999999999998e-07 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339158+0000", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.666e-06 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339164+0000", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2889999999999997e-06 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339167+0000", 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8249999999999998e-06 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383147+0000", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043979964000000003 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383250+0000", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102815 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383326+0000", 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5939999999999995e-05 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.253 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20729 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 7700480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8320973309999999, 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383923999999998, 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20729 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339172+0000", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.65e-07 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339179+0000", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.387e-06 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339183+0000", 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9090000000000002e-06 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339185+0000", 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7190000000000001e-06 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383369+0000", 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044183275000000001 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383462+0000", 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3814000000000007e-05 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383556+0000", 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3554999999999995e-05 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.258 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:20730 3.1c 3:3a21b42a:::rbd_data.11249227f48c.0000000000000004:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8320799509999999, 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044529740999999998, 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 20730 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339189+0000", 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339190+0000", 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7000000000000001e-07 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339198+0000", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8019999999999998e-06 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339205+0000", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7449999999999998e-06 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.339210+0000", 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9470000000000002e-06 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383584+0000", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044374007 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383657+0000", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2867000000000006e-05 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.383719+0000", 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2102999999999998e-05 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.263 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1573 read 0x6514cd5 thru 0x651a633 (0x595f bytes) 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1574 read 0x480ac52 thru 0x480db10 (0x2ebf bytes) 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1577 read 0xad4a417 thru 0xad51f8b (0x7b75 bytes) 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1578 punch from 0x97104ea to 0x971faa7, (0xf5bd bytes) 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1579 read 0x670dbc9 thru 0x67175f0 (0x9a28 bytes) 2024-09-22T01:14:34.264 INFO:teuthology.orchestra.run.smithi096.stdout:1580 write 0x38a6eaf thru 0x38b4e20 (0xdf72 bytes) 2024-09-22T01:14:34.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:34.266 INFO:teuthology.orchestra.run.smithi096.stdout:1582 write 0xc2d9a68 thru 0xc2e3911 (0x9eaa bytes) 2024-09-22T01:14:34.266 INFO:teuthology.orchestra.run.smithi096.stdout:1584 writesame 0x62ca307 thru 0x62d4214 data_size 0x1(0x9f0e bytes) 2024-09-22T01:14:34.266 INFO:teuthology.orchestra.run.smithi096.stdout:1585 write 0x393b49c thru 0x394486a (0x93cf bytes) 2024-09-22T01:14:34.292 INFO:teuthology.orchestra.run.smithi096.stdout:1586 write 0xe2fca34 thru 0xe30ba18 (0xefe5 bytes) 2024-09-22T01:14:34.292 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:34.291+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:34.300 INFO:teuthology.orchestra.run.smithi096.stdout:1587 punch from 0x1eb2d16 to 0x1eb3464, (0x74e bytes) 2024-09-22T01:14:34.300 INFO:teuthology.orchestra.run.smithi096.stdout:1589 read 0x8ba331b thru 0x8ba7d16 (0x49fc bytes) 2024-09-22T01:14:34.301 INFO:teuthology.orchestra.run.smithi096.stdout:1591 write 0xd96fde3 thru 0xd97dce7 (0xdf05 bytes) 2024-09-22T01:14:34.307 INFO:teuthology.orchestra.run.smithi096.stdout:1592 punch from 0x381b539 to 0x38232c6, (0x7d8d bytes) 2024-09-22T01:14:34.319 INFO:teuthology.orchestra.run.smithi096.stdout:1593 trunc from 0xe30ba19 to 0x8066563 2024-09-22T01:14:34.341 INFO:teuthology.orchestra.run.smithi096.stdout:1595 read 0x4275c73 thru 0x4276c2e (0xfbc bytes) 2024-09-22T01:14:34.342 INFO:teuthology.orchestra.run.smithi096.stdout:1598 write 0x20e1056 thru 0x20ecbb4 (0xbb5f bytes) 2024-09-22T01:14:34.346 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:34.346 INFO:teuthology.orchestra.run.smithi096.stdout:1600 read 0x44a762f thru 0x44a8b59 (0x152b bytes) 2024-09-22T01:14:34.353 INFO:teuthology.orchestra.run.smithi096.stdout:1601 trunc from 0x8066563 to 0xc631411 2024-09-22T01:14:34.360 INFO:teuthology.orchestra.run.smithi096.stdout:1602 read 0x13b46ac thru 0x13bd16a (0x8abf bytes) 2024-09-22T01:14:34.361 INFO:teuthology.orchestra.run.smithi096.stdout:1603 clone 16 order 24 su 2097152 sc 12 2024-09-22T01:14:34.370 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:34.367+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:34.424 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:34.424 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:34.424 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:34.424 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:34.438 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:34.479 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:34.479+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:34.570 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:34.569+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:34.608 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:34.608 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:34.608 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:34.609 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:34.609 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:34.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:34.671 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:34.671+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738857416999998, 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:34.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738836748000001, 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:34.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738806435000001, 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738787180999999, 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:34.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738769079000001, 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:34.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738748355999999, 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738727867000001, 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:34.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:34.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:34.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738706425, 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:34.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738675577999999, 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:34.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:34.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738638051999999, 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:34.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738605924000002, 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:34.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:34.798 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.738582916999999, 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:34.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:34.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.699269084000001, 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:34.803 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:34.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:34.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:34.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.697981903999999, 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:34.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:34.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:34.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.697637721, 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:34.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:34.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:15337 3.e 3:73ab052b:::rbd_data.1124645eb891.0000000000000038:c [sparse-read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1997781920000001, 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017128462000000001, 2024-09-22T01:14:34.819 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15337 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546210+0000", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546215+0000", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3800000000000002e-06 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546219+0000", 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0620000000000002e-06 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554742+0000", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085230519999999997 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554860+0000", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117883 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.563336+0000", 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084759050000000006 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:21056 3.e 3:70da9618:::rbd_object_map.11247e2569dd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8463068280000003, 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014372493, 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 21056 2024-09-22T01:14:34.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899680+0000", 2024-09-22T01:14:34.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899683+0000", 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5669999999999999e-06 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899690+0000", 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.776e-06 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899696+0000", 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7790000000000001e-06 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911432+0000", 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011736394000000001 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911543+0000", 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110684 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911906+0000", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036337899999999999 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913199+0000", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012926000000000001 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913966+0000", 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076740200000000004 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914016+0000", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039e-05 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914052+0000", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5404999999999997e-05 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22363 3.5 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4642914089999999, 2024-09-22T01:14:34.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.025573505, 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22363 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281698+0000", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1769999999999998e-06 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285440+0000", 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037427089999999999 2024-09-22T01:14:34.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285474+0000", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3395000000000001e-05 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285490+0000", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6059e-05 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285579+0000", 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8869000000000006e-05 2024-09-22T01:14:34.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285746+0000", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167412 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.286076+0000", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000330052 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.297638+0000", 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011562227 2024-09-22T01:14:34.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307056+0000", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094173729999999997 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307155+0000", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8830000000000001e-05 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307268+0000", 2024-09-22T01:14:34.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113402 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:22368 3.1e e48/24 3:7c2af90c:::rbd_data.1124dfca90b7.0000000000000006:head v 48'53, mlcod=46'51)", 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4436113600000002, 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018959536999999999, 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.309944+0000", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075689340000000003 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.309979+0000", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5136999999999999e-05 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310002+0000", 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2884999999999999e-05 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310151+0000", 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014858100000000001 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310202+0000", 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0725e-05 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.321204+0000", 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011002236 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.321334+0000", 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013039900000000001 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:22367 3.14 e48/28 3:2d346ba6:::rbd_data.1124dfca90b7.0000000000000007:head v 48'118, mlcod=47'116)", 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.4435535279999998, 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016597872999999999, 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302434+0000", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2750000000000001e-06 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305003+0000", 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025690840000000001 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305030+0000", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7219000000000001e-05 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305043+0000", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3253e-05 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305165+0000", 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121815 2024-09-22T01:14:34.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305213+0000", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7426000000000003e-05 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.318866+0000", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013653099 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.319030+0000", 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164702 2024-09-22T01:14:34.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:34.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:34.844 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:34.845 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:34.846 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:34.843+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:34.946 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:34.944+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3.194aa6f4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.447532551, 2024-09-22T01:14:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.447578368, 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:14:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-22T01:14:35.019 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:35.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:35.075 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:35.075+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:35.145 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:35.145 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:35.145 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:35.145 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:35.146 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:35.148 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:35.147+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:35.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:35.249 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:35.248+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.302191102000002, 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2200 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7110656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297625554, 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046602256000000002, 2024-09-22T01:14:35.331 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2200 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024420+0000", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024422+0000", 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024429+0000", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024434+0000", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3850000000000001e-06 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070876+0000", 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046442215000000002 2024-09-22T01:14:35.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.070950+0000", 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3189999999999996e-05 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071022+0000", 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2273000000000004e-05 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2201 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.29760306, 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046753765000000003, 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2201 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024445+0000", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3489999999999999e-06 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024451+0000", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4470000000000003e-06 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024457+0000", 2024-09-22T01:14:35.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071061+0000", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046603415000000002 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071123+0000", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2509999999999996e-05 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071196+0000", 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2719000000000006e-05 2024-09-22T01:14:35.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2202 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297580064999998, 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046876332, 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.340 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2202 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024468+0000", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024474+0000", 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2024-09-22T01:14:35.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024480+0000", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071230+0000", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046749717000000003 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071298+0000", 2024-09-22T01:14:35.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8266e-05 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071341+0000", 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3350000000000003e-05 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2203 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297557842, 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046984537999999999, 2024-09-22T01:14:35.344 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2203 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024488+0000", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024490+0000", 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024497+0000", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9970000000000001e-06 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024503+0000", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.31e-06 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071374+0000", 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046871082000000001 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071423+0000", 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9172999999999998e-05 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071472+0000", 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8973e-05 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.348 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2211 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297535237000002, 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124055999999998, 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2211 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3399999999999999e-07 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024513+0000", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024520+0000", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7410000000000003e-06 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024533+0000", 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2867999999999999e-05 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071509+0000", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046976047 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071569+0000", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0482999999999997e-05 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071634+0000", 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4834999999999998e-05 2024-09-22T01:14:35.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2212 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297504404000001, 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047215269999999997, 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2212 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3e-07 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024544+0000", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0690000000000002e-06 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024551+0000", 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.686e-06 2024-09-22T01:14:35.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024557+0000", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.806e-06 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071661+0000", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047104241999999998 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071703+0000", 2024-09-22T01:14:35.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071756+0000", 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3171999999999998e-05 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2213 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297469309, 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047299709000000002, 2024-09-22T01:14:35.357 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2213 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024593+0000", 2024-09-22T01:14:35.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6223e-05 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024600+0000", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024607+0000", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080000000000003e-06 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071783+0000", 2024-09-22T01:14:35.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047175863999999998 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071822+0000", 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071875+0000", 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3801999999999998e-05 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.361 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2214 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.297429432000001, 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047393246999999999, 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2214 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026492+0000", 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018767059999999999 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026500+0000", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026513+0000", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3422e-05 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026523+0000", 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071904+0000", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045380993000000001 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071942+0000", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572999999999998e-05 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.072009+0000", 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6797000000000004e-05 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2405 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 4915200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292906531, 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04671264, 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.366 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2405 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029140+0000", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1209999999999999e-06 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029143+0000", 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2049999999999998e-06 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029146+0000", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7800000000000001e-06 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075719+0000", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046573497999999998 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075787+0000", 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7135e-05 2024-09-22T01:14:35.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075851+0000", 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4660999999999996e-05 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2406 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292895588, 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046865920999999998, 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2406 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.22e-07 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029151+0000", 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029154+0000", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.439e-06 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029158+0000", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075881+0000", 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046723187999999999 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075951+0000", 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9565000000000002e-05 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076015+0000", 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4931000000000006e-05 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2407 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292883372999999, 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047019894, 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2407 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:35.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3900000000000001e-07 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029163+0000", 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2190000000000001e-06 2024-09-22T01:14:35.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029167+0000", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3390000000000001e-06 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029169+0000", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9249999999999999e-06 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076045+0000", 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046875173999999999 2024-09-22T01:14:35.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076114+0000", 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.902e-05 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076182+0000", 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7978000000000005e-05 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2408 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292871870999999, 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047176774999999997, 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2408 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.35e-07 2024-09-22T01:14:35.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029175+0000", 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029178+0000", 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2650000000000001e-06 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029181+0000", 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6860000000000002e-06 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076216+0000", 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035615000000003 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076286+0000", 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9680999999999999e-05 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076350+0000", 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4172999999999997e-05 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2409 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292860874999999, 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047331204000000002, 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2409 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-07 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029186+0000", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029190+0000", 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1169999999999997e-06 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029194+0000", 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5310000000000003e-06 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076377+0000", 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047182737000000002 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076447+0000", 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0411000000000001e-05 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076515+0000", 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7960999999999995e-05 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.387 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2410 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292828804999999, 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047479322999999997, 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2410 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029217+0000", 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029219+0000", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.339e-06 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029226+0000", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6429999999999998e-06 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029231+0000", 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.152e-06 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076544+0000", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047312694000000002 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076630+0000", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5753000000000004e-05 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076696+0000", 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6087999999999995e-05 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2411 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292799483, 2024-09-22T01:14:35.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047606067000000002, 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2411 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029249+0000", 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4739999999999999e-06 2024-09-22T01:14:35.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029256+0000", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999996e-06 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029260+0000", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076724+0000", 2024-09-22T01:14:35.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047463678000000002 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076786+0000", 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000005e-05 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076852+0000", 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5759999999999994e-05 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2412 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292777347000001, 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047679747000000001, 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2412 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:35.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029270+0000", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0540000000000002e-06 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029277+0000", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029282+0000", 2024-09-22T01:14:35.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6890000000000003e-06 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076869+0000", 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047587741000000003 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076909+0000", 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9504e-05 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076947+0000", 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8578999999999998e-05 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2413 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292755248999999, 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047815037999999997, 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2413 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029292+0000", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029299+0000", 2024-09-22T01:14:35.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029304+0000", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076974+0000", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047670055000000003 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077036+0000", 2024-09-22T01:14:35.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2019000000000004e-05 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077105+0000", 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9127000000000004e-05 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2414 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.292734519, 2024-09-22T01:14:35.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048254547000000002, 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2414 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029313+0000", 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.075e-06 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029320+0000", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029325+0000", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077133+0000", 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047808311999999999 2024-09-22T01:14:35.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077197+0000", 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3912999999999996e-05 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077565+0000", 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830000000000001 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3931 3.11 3:8f505f18:::rbd_data.1124ef2748ce.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.206866324, 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048441529999999997, 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3931 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115182+0000", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115189+0000", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115194+0000", 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160040+0000", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044845812999999998 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160142+0000", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010177499999999999 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160287+0000", 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014540899999999999 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163457+0000", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031691570000000001 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163508+0000", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1944000000000003e-05 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163581+0000", 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2123e-05 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:35.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163620+0000", 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709000000000002e-05 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:35.416 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:35.417 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:35.418 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:35.416+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:35.518 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:35.516+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:35.555 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:35.556 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:35.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:35.619 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:35.619+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:35.651 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone15 from 0xc631411 (overlap 0x8066563) to 0x19ae113 2024-09-22T01:14:35.720 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:35.719+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:35.743 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:35.743 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:35.743 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:35.743 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:35.743 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:35.757 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:35.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:14:35.847 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:35.847+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:35.861 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-22T01:14:35.863 INFO:teuthology.orchestra.run.smithi096.stdout:1604 write 0x1438708 thru 0x143e76b (0x6064 bytes) 2024-09-22T01:14:35.878 INFO:teuthology.orchestra.run.smithi096.stdout:1605 writesame 0x2a15bae thru 0x2a1cc1f data_size 0x25(0x7072 bytes) 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5341886020000004, 2024-09-22T01:14:35.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:35.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5341224359999996, 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.891 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:35.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.895 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5340623989999997, 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5340273370000004, 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:35.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5339970669999996, 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5339674939999997, 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:35.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5339134029999997, 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5338575590000003, 2024-09-22T01:14:35.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:35.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:35.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5337996690000004, 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5337379359999996, 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:35.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.930 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5336424730000005, 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:35.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:35.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:35.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5335806500000002, 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:35.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:35.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5334895099999999, 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:35.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:35.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:35.945 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5334276039999999, 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:35.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:35.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5333671170000001, 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:35.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5333096470000003, 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:35.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5332523670000002, 2024-09-22T01:14:35.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:35.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5331928430000001, 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:35.969 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5331399589999997, 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:35.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:35.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.5240273970000002, 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:35.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:35.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:35.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:35.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:35.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:35.983 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:35.985 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:35.986 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:35.984+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:1606 write 0x7741245 thru 0x774f8c9 (0xe685 bytes) 2024-09-22T01:14:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:1607 write 0x5684b33 thru 0x568b584 (0x6a52 bytes) 2024-09-22T01:14:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:1608 punch from 0x368e63e to 0x369a641, (0xc003 bytes) 2024-09-22T01:14:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:1609 write 0x7c526b8 thru 0x7c55f8f (0x38d8 bytes) 2024-09-22T01:14:35.986 INFO:teuthology.orchestra.run.smithi096.stdout:1610 trunc from 0xc631411 to 0xda50a95 2024-09-22T01:14:35.987 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:35.987 INFO:teuthology.orchestra.run.smithi096.stdout:1612 write 0xebe3e67 thru 0xebe7e65 (0x3fff bytes) 2024-09-22T01:14:35.987 INFO:teuthology.orchestra.run.smithi096.stdout:1613 trunc from 0xebe7e66 to 0x607cda1 2024-09-22T01:14:36.031 INFO:teuthology.orchestra.run.smithi096.stdout:1614 read 0xcc161d thru 0xcc8d6a (0x774e bytes) 2024-09-22T01:14:36.031 INFO:teuthology.orchestra.run.smithi096.stdout:1615 read 0x21f9e20 thru 0x21fab84 (0xd65 bytes) 2024-09-22T01:14:36.032 INFO:teuthology.orchestra.run.smithi096.stdout:1617 write 0x55b9dba thru 0x55c029e (0x64e5 bytes) 2024-09-22T01:14:36.038 INFO:teuthology.orchestra.run.smithi096.stdout:1618 trunc from 0x607cda1 to 0x45add08 2024-09-22T01:14:36.052 INFO:teuthology.orchestra.run.smithi096.stdout:1619 write 0x1a4345e thru 0x1a44d4d (0x18f0 bytes) 2024-09-22T01:14:36.054 INFO:teuthology.orchestra.run.smithi096.stdout:1620 read 0x3655149 thru 0x3661163 (0xc01b bytes) 2024-09-22T01:14:36.055 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.055 INFO:teuthology.orchestra.run.smithi096.stdout:1624 punch from 0x11d5e67 to 0x11d6d73, (0xf0c bytes) 2024-09-22T01:14:36.055 INFO:teuthology.orchestra.run.smithi096.stdout:1625 punch from 0x2ffc62e to 0x30084a3, (0xbe75 bytes) 2024-09-22T01:14:36.056 INFO:teuthology.orchestra.run.smithi096.stdout:1628 punch from 0x124db13 to 0x12569aa, (0x8e97 bytes) 2024-09-22T01:14:36.056 INFO:teuthology.orchestra.run.smithi096.stdout:1629 write 0x237920f thru 0x238691f (0xd711 bytes) 2024-09-22T01:14:36.058 INFO:teuthology.orchestra.run.smithi096.stdout:1631 read 0x345ae34 thru 0x3463657 (0x8824 bytes) 2024-09-22T01:14:36.068 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:36.072 INFO:teuthology.orchestra.run.smithi096.stdout:1632 write 0x5e60b4a thru 0x5e6179d (0xc54 bytes) 2024-09-22T01:14:36.081 INFO:teuthology.orchestra.run.smithi096.stdout:1635 trunc from 0x5e6179e to 0x8ea1221 2024-09-22T01:14:36.085 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:36.084+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:36.103 INFO:teuthology.orchestra.run.smithi096.stdout:1636 writesame 0xb12ff82 thru 0xb130f7a data_size 0x1d(0xff9 bytes) 2024-09-22T01:14:36.106 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.113 INFO:teuthology.orchestra.run.smithi096.stdout:1639 trunc from 0xb130f7b to 0xee06720 2024-09-22T01:14:36.122 INFO:teuthology.orchestra.run.smithi096.stdout:1640 writesame 0x9098d94 thru 0x909fd6b data_size 0xc(0x6fd8 bytes) 2024-09-22T01:14:36.127 INFO:teuthology.orchestra.run.smithi096.stdout:1641 write 0x18e32bc thru 0x18ee870 (0xb5b5 bytes) 2024-09-22T01:14:36.130 INFO:teuthology.orchestra.run.smithi096.stdout:1643 write 0x7a3b627 thru 0x7a3d030 (0x1a0a bytes) 2024-09-22T01:14:36.133 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.155 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:36.155 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:36.155 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:36.155 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:36.156 INFO:teuthology.orchestra.run.smithi096.stdout:1647 trunc from 0xee06720 to 0x4ac95e 2024-09-22T01:14:36.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:36.185 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:36.183+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:36.199 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.199 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.199 INFO:teuthology.orchestra.run.smithi096.stdout:1651 read 0xb6684 thru 0xbfa8d (0x940a bytes) 2024-09-22T01:14:36.205 INFO:teuthology.orchestra.run.smithi096.stdout:1652 trunc from 0x4ac95e to 0x3b8daf9 2024-09-22T01:14:36.230 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:36.230 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:36.232 INFO:teuthology.orchestra.run.smithi096.stdout:1655 writesame 0xc9e5c4a thru 0xc9f4b84 data_size 0xd(0xef3b bytes) 2024-09-22T01:14:36.232 INFO:teuthology.orchestra.run.smithi096.stdout:1659 writesame 0x8af14e9 thru 0x8af49ab data_size 0x1(0x34c3 bytes) 2024-09-22T01:14:36.242 INFO:teuthology.orchestra.run.smithi096.stdout:1661 write 0xb87d295 thru 0xb8830ce (0x5e3a bytes) 2024-09-22T01:14:36.249 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:36.249 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:36.262 INFO:teuthology.orchestra.run.smithi096.stdout:1662 trunc from 0xc9f4b85 to 0x1a4254e 2024-09-22T01:14:36.266 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:36.267 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:36.277 INFO:teuthology.orchestra.run.smithi096.stdout:1663 punch from 0x20bd4d to 0x20c63f, (0x8f2 bytes) 2024-09-22T01:14:36.277 INFO:teuthology.orchestra.run.smithi096.stdout:1665 read 0x13b22 thru 0x148cc (0xdab bytes) 2024-09-22T01:14:36.284 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:36.285 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:36.286 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:36.283+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:36.300 INFO:teuthology.orchestra.run.smithi096.stdout:1666 writesame 0xd292383 thru 0xd2937c3 data_size 0x11(0x1441 bytes) 2024-09-22T01:14:36.300 INFO:teuthology.orchestra.run.smithi096.stdout:1667 read 0x22de39b thru 0x22e12cd (0x2f33 bytes) 2024-09-22T01:14:36.300 INFO:teuthology.orchestra.run.smithi096.stdout:1670 read 0x94e165 thru 0x95bb48 (0xd9e4 bytes) 2024-09-22T01:14:36.301 INFO:teuthology.orchestra.run.smithi096.stdout:1672 read 0x4f55e6 thru 0x501415 (0xbe30 bytes) 2024-09-22T01:14:36.301 INFO:teuthology.orchestra.run.smithi096.stdout:1673 write 0x43b16eb thru 0x43bcf85 (0xb89b bytes) 2024-09-22T01:14:36.303 INFO:teuthology.orchestra.run.smithi096.stdout:1675 read 0x4494e01 thru 0x44962ae (0x14ae bytes) 2024-09-22T01:14:36.303 INFO:teuthology.orchestra.run.smithi096.stdout:1676 write 0xd2584a8 thru 0xd267585 (0xf0de bytes) 2024-09-22T01:14:36.305 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:36.308 INFO:teuthology.orchestra.run.smithi096.stdout:1677 write 0x94e97fc thru 0x94f64d6 (0xccdb bytes) 2024-09-22T01:14:36.311 INFO:teuthology.orchestra.run.smithi096.stdout:1679 writesame 0x1ef35e7 thru 0x1ef8ace data_size 0x1a(0x54e8 bytes) 2024-09-22T01:14:36.315 INFO:teuthology.orchestra.run.smithi096.stdout:1680 write 0x56dbb21 thru 0x56e1dc4 (0x62a4 bytes) 2024-09-22T01:14:36.320 INFO:teuthology.orchestra.run.smithi096.stdout:1681 read 0x431bbee thru 0x431d798 (0x1bab bytes) 2024-09-22T01:14:36.321 INFO:teuthology.orchestra.run.smithi096.stdout:1684 writesame 0x34bab29 thru 0x34ca455 data_size 0xb(0xf92d bytes) 2024-09-22T01:14:36.322 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:36.323 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:36.326 INFO:teuthology.orchestra.run.smithi096.stdout:1686 writesame 0xa6eb8fa thru 0xa6f3c6d data_size 0x4(0x8374 bytes) 2024-09-22T01:14:36.334 INFO:teuthology.orchestra.run.smithi096.stdout:1687 read 0x1a885d0 thru 0x1a93743 (0xb174 bytes) 2024-09-22T01:14:36.334 INFO:teuthology.orchestra.run.smithi096.stdout:1688 write 0x44613a8 thru 0x4462296 (0xeef bytes) 2024-09-22T01:14:36.338 INFO:teuthology.orchestra.run.smithi096.stdout:1689 write 0x84cd4b8 thru 0x84d49af (0x74f8 bytes) 2024-09-22T01:14:36.343 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:36.344 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:36.346 INFO:teuthology.orchestra.run.smithi096.stdout:1691 trunc from 0xd2937c4 to 0xadc7db0 2024-09-22T01:14:36.354 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:36.354 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:36.365 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:36.365 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:36.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:36.372 INFO:teuthology.orchestra.run.smithi096.stdout:1692 trunc from 0xadc7db0 to 0x7231e71 2024-09-22T01:14:36.385 INFO:teuthology.orchestra.run.smithi096.stdout:1694 write 0xc8b9bf thru 0xc93830 (0x7e72 bytes) 2024-09-22T01:14:36.396 INFO:teuthology.orchestra.run.smithi096.stdout:1695 trunc from 0x7231e71 to 0x8b0f0d 2024-09-22T01:14:36.416 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:36.416+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:36.455 INFO:teuthology.orchestra.run.smithi096.stdout:1697 write 0xb07868b thru 0xb084b10 (0xc486 bytes) 2024-09-22T01:14:36.460 INFO:teuthology.orchestra.run.smithi096.stdout:1699 writesame 0x1ab4365 thru 0x1abb976 data_size 0x2(0x7612 bytes) 2024-09-22T01:14:36.479 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.479 INFO:teuthology.orchestra.run.smithi096.stdout:1704 read 0x1e7d299 thru 0x1e88935 (0xb69d bytes) 2024-09-22T01:14:36.486 INFO:teuthology.orchestra.run.smithi096.stdout:1705 trunc from 0xb084b11 to 0xdad29f5 2024-09-22T01:14:36.488 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:36.488+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:36.497 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:36.498 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:36.498 INFO:teuthology.orchestra.run.smithi096.stdout:1706 read 0x716fc0d thru 0x7173472 (0x3866 bytes) 2024-09-22T01:14:36.499 INFO:teuthology.orchestra.run.smithi096.stdout:1707 read 0xb16aaa1 thru 0xb17746c (0xc9cc bytes) 2024-09-22T01:14:36.500 INFO:teuthology.orchestra.run.smithi096.stdout:1708 punch from 0x17d27db to 0x17dc538, (0x9d5d bytes) 2024-09-22T01:14:36.501 INFO:teuthology.orchestra.run.smithi096.stdout:1710 writesame 0x8ab6ec8 thru 0x8ac084f data_size 0x44(0x9988 bytes) 2024-09-22T01:14:36.509 INFO:teuthology.orchestra.run.smithi096.stdout:1712 writesame 0xcb1c17a thru 0xcb20d8d data_size 0x12(0x4c14 bytes) 2024-09-22T01:14:36.515 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:36.519 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:14:36.523 INFO:teuthology.orchestra.run.smithi096.stdout:1715 read 0x53da42 thru 0x540770 (0x2d2f bytes) 2024-09-22T01:14:36.524 INFO:teuthology.orchestra.run.smithi096.stdout:1716 trunc from 0xdad29f5 to 0xdacbf9c 2024-09-22T01:14:36.561 INFO:teuthology.orchestra.run.smithi096.stdout:1718 trunc from 0xdacbf9c to 0x41636a 2024-09-22T01:14:36.589 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:36.587+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:36.596 INFO:teuthology.orchestra.run.smithi096.stdout:1719 trunc from 0x41636a to 0x24fdec7 2024-09-22T01:14:36.607 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.607 INFO:teuthology.orchestra.run.smithi096.stdout:1721 read 0x2094065 thru 0x209c19b (0x8137 bytes) 2024-09-22T01:14:36.624 INFO:teuthology.orchestra.run.smithi096.stdout:1723 write 0x61fee35 thru 0x620c4ae (0xd67a bytes) 2024-09-22T01:14:36.627 INFO:teuthology.orchestra.run.smithi096.stdout:1724 trunc from 0x620c4af to 0x66c2a4a 2024-09-22T01:14:36.634 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.650 INFO:teuthology.orchestra.run.smithi096.stdout:1726 trunc from 0x66c2a4a to 0xdd5bbbd 2024-09-22T01:14:36.663 INFO:teuthology.orchestra.run.smithi096.stdout:1727 punch from 0xc869525 to 0xc874d79, (0xb854 bytes) 2024-09-22T01:14:36.663 INFO:teuthology.orchestra.run.smithi096.stdout:1729 writesame 0xcd6a0bc thru 0xcd77aa4 data_size 0x5(0xd9e9 bytes) 2024-09-22T01:14:36.684 INFO:teuthology.orchestra.run.smithi096.stdout:1730 write 0xbcfc1dd thru 0xbcfd51e (0x1342 bytes) 2024-09-22T01:14:36.689 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:36.687+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:36.690 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:36.690 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:36.692 INFO:teuthology.orchestra.run.smithi096.stdout:1731 read 0xadac25c thru 0xadb8b4b (0xc8f0 bytes) 2024-09-22T01:14:36.693 INFO:teuthology.orchestra.run.smithi096.stdout:1732 writesame 0xd24da6c thru 0xd250059 data_size 0x5(0x25ee bytes) 2024-09-22T01:14:36.702 INFO:teuthology.orchestra.run.smithi096.stdout:1734 punch from 0x659729 to 0x66598b, (0xc262 bytes) 2024-09-22T01:14:36.703 INFO:teuthology.orchestra.run.smithi096.stdout:1736 writesame 0x19f212e thru 0x19fc93e data_size 0x5(0xa811 bytes) 2024-09-22T01:14:36.705 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:36.719 INFO:teuthology.orchestra.run.smithi096.stdout:1738 read 0x1f83441 thru 0x1f8d86b (0xa42b bytes) 2024-09-22T01:14:36.719 INFO:teuthology.orchestra.run.smithi096.stdout:1739 write 0xa9d2d8f thru 0xa9d83ae (0x5620 bytes) 2024-09-22T01:14:36.723 INFO:teuthology.orchestra.run.smithi096.stdout:1740 write 0x4d74cc7 thru 0x4d80f70 (0xc2aa bytes) 2024-09-22T01:14:36.727 INFO:teuthology.orchestra.run.smithi096.stdout:1741 writesame 0x60355d7 thru 0x603f840 data_size 0x1(0xa26a bytes) 2024-09-22T01:14:36.782 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:36.782 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:36.782 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-22T01:14:36.782 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-22T01:14:36.790 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:36.788+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:36.800 INFO:teuthology.orchestra.run.smithi096.stdout:1742 writesame 0x6d59f6 thru 0x6e3201 data_size 0x42(0xd80c bytes) 2024-09-22T01:14:36.826 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:36.827 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:36.827 INFO:teuthology.orchestra.run.smithi096.stdout:1743 trunc from 0xdd5bbbd to 0x5b803bc 2024-09-22T01:14:36.843 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:36.856 INFO:teuthology.orchestra.run.smithi096.stdout:1744 read 0xa462cb thru 0xa530bf (0xcdf5 bytes) 2024-09-22T01:14:36.856 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.885 INFO:teuthology.orchestra.run.smithi096.stdout:1746 write 0xea701d8 thru 0xea7613c (0x5f65 bytes) 2024-09-22T01:14:36.891 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:36.890+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:36.893 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.893 INFO:teuthology.orchestra.run.smithi096.stdout:1749 write 0x600b1c7 thru 0x6012f4f (0x7d89 bytes) 2024-09-22T01:14:36.899 INFO:teuthology.orchestra.run.smithi096.stdout:1750 trunc from 0xea7613d to 0xd2c3d9a 2024-09-22T01:14:36.912 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:36.912 INFO:teuthology.orchestra.run.smithi096.stdout:1753 write 0x425e83 thru 0x42b727 (0x58a5 bytes) 2024-09-22T01:14:36.915 INFO:teuthology.orchestra.run.smithi096.stdout:1754 write 0xae0504b thru 0xae11a79 (0xca2f bytes) 2024-09-22T01:14:36.918 INFO:teuthology.orchestra.run.smithi096.stdout:1755 read 0x92a6d26 thru 0x92b3393 (0xc66e bytes) 2024-09-22T01:14:36.935 INFO:teuthology.orchestra.run.smithi096.stdout:1756 trunc from 0xd2c3d9a to 0x28b610b 2024-09-22T01:14:36.950 INFO:teuthology.orchestra.run.smithi096.stdout:1760 read 0x232ebed thru 0x233da5e (0xee72 bytes) 2024-09-22T01:14:36.950 INFO:teuthology.orchestra.run.smithi096.stdout:1761 read 0x1a7fb09 thru 0x1a882c2 (0x87ba bytes) 2024-09-22T01:14:36.951 INFO:teuthology.orchestra.run.smithi096.stdout:1762 write 0x3b1747 thru 0x3bd632 (0xbeec bytes) 2024-09-22T01:14:36.953 INFO:teuthology.orchestra.run.smithi096.stdout:1765 read 0xe92f1e thru 0xe93bc6 (0xca9 bytes) 2024-09-22T01:14:36.976 INFO:teuthology.orchestra.run.smithi096.stdout:1767 write 0xcfc86c1 thru 0xcfd4275 (0xbbb5 bytes) 2024-09-22T01:14:36.981 INFO:teuthology.orchestra.run.smithi096.stdout:1769 write 0xa0f64e7 thru 0xa0f7286 (0xda0 bytes) 2024-09-22T01:14:36.992 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:36.992+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:37.001 INFO:teuthology.orchestra.run.smithi096.stdout:1770 trunc from 0xcfd4276 to 0x14afa0b 2024-09-22T01:14:37.019 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.019 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:37.043 INFO:teuthology.orchestra.run.smithi096.stdout:1771 write 0x6673410 thru 0x6681337 (0xdf28 bytes) 2024-09-22T01:14:37.045 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.064 INFO:teuthology.orchestra.run.smithi096.stdout:1773 write 0xb9d55f0 thru 0xb9dddf0 (0x8801 bytes) 2024-09-22T01:14:37.067 INFO:teuthology.orchestra.run.smithi096.stdout:1774 writesame 0x60e3b83 thru 0x60e6c09 data_size 0x29(0x3087 bytes) 2024-09-22T01:14:37.070 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.070 INFO:teuthology.orchestra.run.smithi096.stdout:1776 read 0x14e0d2d thru 0x14e8f33 (0x8207 bytes) 2024-09-22T01:14:37.083 INFO:teuthology.orchestra.run.smithi096.stdout:1777 write 0xcc2e882 thru 0xcc39d16 (0xb495 bytes) 2024-09-22T01:14:37.090 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.093 INFO:teuthology.orchestra.run.smithi096.stdout:1779 trunc from 0xcc39d17 to 0xe427581 2024-09-22T01:14:37.102 INFO:teuthology.orchestra.run.smithi096.stdout:1780 writesame 0x54485ba thru 0x54500b3 data_size 0x63(0x7afa bytes) 2024-09-22T01:14:37.107 INFO:teuthology.orchestra.run.smithi096.stdout:1781 write 0xc8b5ca1 thru 0xc8ba6a0 (0x4a00 bytes) 2024-09-22T01:14:37.112 INFO:teuthology.orchestra.run.smithi096.stdout:1782 write 0xcd335d0 thru 0xcd3424e (0xc7f bytes) 2024-09-22T01:14:37.115 INFO:teuthology.orchestra.run.smithi096.stdout:1783 read 0x2cfc308 thru 0x2cfca7f (0x778 bytes) 2024-09-22T01:14:37.116 INFO:teuthology.orchestra.run.smithi096.stdout:1784 writesame 0x2d3b57 thru 0x2dd328 data_size 0x2(0x97d2 bytes) 2024-09-22T01:14:37.120 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:37.120+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:37.149 INFO:teuthology.orchestra.run.smithi096.stdout:1786 write 0x2a5a001 thru 0x2a5f70a (0x570a bytes) 2024-09-22T01:14:37.155 INFO:teuthology.orchestra.run.smithi096.stdout:1787 writesame 0x361ca51 thru 0x362316f data_size 0x1(0x671f bytes) 2024-09-22T01:14:37.158 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-22T01:14:37.154+0000 7f33f656f640 -1 mon.a@0(leader).osd e54 definitely_dead 0 2024-09-22T01:14:37.194 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:37.192+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:37.202 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.202 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.206 INFO:teuthology.orchestra.run.smithi096.stdout:1791 trunc from 0xe427581 to 0x8df157a 2024-09-22T01:14:37.217 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:37.265 INFO:teuthology.orchestra.run.smithi096.stdout:1792 writesame 0xbef9e2b thru 0xbf02882 data_size 0x4c(0x8a58 bytes) 2024-09-22T01:14:37.265 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.278 INFO:teuthology.orchestra.run.smithi096.stdout:1796 trunc from 0xbf02883 to 0x6bdf38d 2024-09-22T01:14:37.295 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:37.290+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:37.299 INFO:teuthology.orchestra.run.smithi096.stdout:1798 punch from 0x957cee to 0x960068, (0x837a bytes) 2024-09-22T01:14:37.299 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.300 INFO:teuthology.orchestra.run.smithi096.stdout:1800 trunc from 0x6bdf38d to 0x64d2089 2024-09-22T01:14:37.314 INFO:teuthology.orchestra.run.smithi096.stdout:1801 read 0x311e5a3 thru 0x3128ad0 (0xa52e bytes) 2024-09-22T01:14:37.315 INFO:teuthology.orchestra.run.smithi096.stdout:1802 punch from 0x45fae67 to 0x460990b, (0xeaa4 bytes) 2024-09-22T01:14:37.315 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.316 INFO:teuthology.orchestra.run.smithi096.stdout:1804 read 0x134d53c thru 0x1352559 (0x501e bytes) 2024-09-22T01:14:37.316 INFO:teuthology.orchestra.run.smithi096.stdout:1805 read 0x45b2649 thru 0x45bd30d (0xacc5 bytes) 2024-09-22T01:14:37.336 INFO:teuthology.orchestra.run.smithi096.stdout:1806 write 0x952a2b3 thru 0x952ddb7 (0x3b05 bytes) 2024-09-22T01:14:37.364 INFO:teuthology.orchestra.run.smithi096.stdout:1807 write 0xe7d3f58 thru 0xe7da4cd (0x6576 bytes) 2024-09-22T01:14:37.371 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:37.371 INFO:teuthology.orchestra.run.smithi096.stdout:1809 write 0xc1331ec thru 0xc134ef3 (0x1d08 bytes) 2024-09-22T01:14:37.378 INFO:teuthology.orchestra.run.smithi096.stdout:1810 read 0x5050c2d thru 0x505d878 (0xcc4c bytes) 2024-09-22T01:14:37.378 INFO:teuthology.orchestra.run.smithi096.stdout:1811 write 0x6740274 thru 0x6741889 (0x1616 bytes) 2024-09-22T01:14:37.381 INFO:teuthology.orchestra.run.smithi096.stdout:1812 writesame 0x73d6a24 thru 0x73d9fbd data_size 0x3(0x359a bytes) 2024-09-22T01:14:37.385 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.385 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.391 INFO:teuthology.orchestra.run.smithi096.stdout:1814 write 0x8276804 thru 0x827fc82 (0x947f bytes) 2024-09-22T01:14:37.395 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:37.394+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:37.399 INFO:teuthology.orchestra.run.smithi096.stdout:1815 read 0x1a46b11 thru 0x1a5633d (0xf82d bytes) 2024-09-22T01:14:37.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:37.401 INFO:teuthology.orchestra.run.smithi096.stdout:1819 writesame 0x4674d05 thru 0x46798f0 data_size 0x4(0x4bec bytes) 2024-09-22T01:14:37.412 INFO:teuthology.orchestra.run.smithi096.stdout:1820 read 0xe4b4ece thru 0xe4b6d4e (0x1e81 bytes) 2024-09-22T01:14:37.413 INFO:teuthology.orchestra.run.smithi096.stdout:1821 write 0x564fc8a thru 0x56525a1 (0x2918 bytes) 2024-09-22T01:14:37.426 INFO:teuthology.orchestra.run.smithi096.stdout:1822 trunc from 0xe7da4ce to 0xbe63984 2024-09-22T01:14:37.454 INFO:teuthology.orchestra.run.smithi096.stdout:1823 read 0xac0cc28 thru 0xac16f35 (0xa30e bytes) 2024-09-22T01:14:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:1825 write 0xe8d374e thru 0xe8da132 (0x69e5 bytes) 2024-09-22T01:14:37.482 INFO:teuthology.orchestra.run.smithi096.stdout:1826 write 0x21b6745 thru 0x21b6fc3 (0x87f bytes) 2024-09-22T01:14:37.490 INFO:teuthology.orchestra.run.smithi096.stdout:1827 write 0x6f58a0 thru 0x6fa9ad (0x510e bytes) 2024-09-22T01:14:37.493 INFO:teuthology.orchestra.run.smithi096.stdout:1829 punch from 0x69d2b48 to 0x69d6834, (0x3cec bytes) 2024-09-22T01:14:37.497 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:37.496+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:37.509 INFO:teuthology.orchestra.run.smithi096.stdout:1830 trunc from 0xe8da133 to 0x94016be 2024-09-22T01:14:37.520 INFO:teuthology.orchestra.run.smithi096.stderr:marked in osd.2. 2024-09-22T01:14:37.526 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.527 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:37.551 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-22T01:14:37.598 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:37.594+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:37.666 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.666 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:37.723 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:37.722+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:37.800 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:37.798+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:37.846 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:37.846 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:37.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:37.901 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:37.900+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:38.002 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:37.998+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:38.029 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:38.029 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:38.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:38.103 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:38.100+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:38.204 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:38.204+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:38.211 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:38.211 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:38.225 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:38.305 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:38.304+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:38.346 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:38.346 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:38.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:38.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:38.407 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:38.406+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:38.479 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:38.479 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:38.494 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:38.541 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:38.538+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:38.548 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:14:38.564 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:38.564 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:38.583 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:38.583 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:38.600 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:38.600 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:38.608 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:38.604+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:38.620 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:38.621 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:38.628 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:38.628 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:38.628 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:38.628 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:38.643 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:38.644 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:38.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:38.666 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:38.694 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:38.694 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:38.708 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:38.706+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:38.717 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:38.790 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:38.790 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:38.790 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:38.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:38.851 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:38.850+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:38.910 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:38.906+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:38.911 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2029 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4358144~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902492477999999, 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014426245000000001, 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2029 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007129+0000", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007131+0000", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0099999999999998e-06 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007138+0000", 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.058e-06 2024-09-22T01:14:38.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007142+0000", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4299999999999999e-06 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021427+0000", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014284702 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021492+0000", 2024-09-22T01:14:38.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5265999999999995e-05 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021555+0000", 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389000000000002e-05 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2030 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4685824~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902471809000001, 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559684, 2024-09-22T01:14:38.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2030 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007149+0000", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007150+0000", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-07 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007155+0000", 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007168+0000", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3010000000000001e-05 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007173+0000", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021581+0000", 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01440789 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021647+0000", 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6364000000000004e-05 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021709+0000", 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1581000000000006e-05 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2031 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 4915200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902441496000002, 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014690943, 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2031 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007180+0000", 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:14:38.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007182+0000", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7999999999999999e-06 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007188+0000", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1510000000000001e-06 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007192+0000", 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5199999999999999e-06 2024-09-22T01:14:38.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021736+0000", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014543416 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021804+0000", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8087e-05 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021870+0000", 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6617000000000003e-05 2024-09-22T01:14:38.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2032 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5242880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902422242, 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844522000000001, 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2032 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007199+0000", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007201+0000", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7579999999999999e-06 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007207+0000", 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-22T01:14:38.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007211+0000", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6179999999999997e-06 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021897+0000", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014685281 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.021965+0000", 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8083999999999994e-05 2024-09-22T01:14:38.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022043+0000", 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8505999999999997e-05 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2033 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 5799936~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902404140000002, 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015007193, 2024-09-22T01:14:38.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2033 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007217+0000", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007219+0000", 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.703e-06 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007225+0000", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9800000000000003e-06 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007229+0000", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4830000000000001e-06 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022074+0000", 2024-09-22T01:14:38.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014844309999999999 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022146+0000", 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2068000000000002e-05 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022224+0000", 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8325000000000006e-05 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2034 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6127616~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902383416999999, 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015172343, 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2034 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007238+0000", 2024-09-22T01:14:38.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007240+0000", 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.683e-06 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007247+0000", 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0010000000000004e-06 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007251+0000", 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022256+0000", 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015004290999999999 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022327+0000", 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1612999999999994e-05 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022410+0000", 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2612000000000001e-05 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2035 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6356992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902362927999999, 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015339892000000001, 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2035 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007258+0000", 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007259+0000", 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007261+0000", 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.916e-06 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007269+0000", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007273+0000", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6920000000000001e-06 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022438+0000", 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015164973 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022518+0000", 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0233000000000002e-05 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022598+0000", 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9546e-05 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2036 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 6684672~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902341486000001, 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015507148, 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2036 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007280+0000", 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007282+0000", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9010000000000001e-06 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007290+0000", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9030000000000004e-06 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007303+0000", 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2845e-05 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022627+0000", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015324910000000001 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022700+0000", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2879999999999993e-05 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022787+0000", 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6353000000000005e-05 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2037 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7241728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902310639, 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015683866000000001, 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2037 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007310+0000", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007311+0000", 2024-09-22T01:14:38.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007313+0000", 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007321+0000", 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9740000000000002e-06 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007340+0000", 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8368000000000001e-05 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022816+0000", 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015475879 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022910+0000", 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4508999999999999e-05 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.022994+0000", 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4153999999999994e-05 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2038 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7569408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902273113, 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015837079, 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2038 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007348+0000", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007358+0000", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.611e-06 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007367+0000", 2024-09-22T01:14:38.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6670000000000002e-06 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007372+0000", 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3240000000000002e-06 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023023+0000", 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015651107000000001 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023102+0000", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9289999999999995e-05 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023185+0000", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2602999999999995e-05 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2039 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 7798784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:38.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902240984999999, 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015964816, 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2039 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:38.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007380+0000", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007381+0000", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6199999999999998e-07 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007383+0000", 2024-09-22T01:14:38.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2529999999999999e-06 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007391+0000", 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1669999999999999e-06 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007396+0000", 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1259999999999997e-06 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023214+0000", 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015817591999999998 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023288+0000", 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4438999999999997e-05 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023345+0000", 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2040 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 8126464~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.902217977999999, 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016059123000000002, 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2040 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007403+0000", 2024-09-22T01:14:38.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007411+0000", 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8120000000000008e-06 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007419+0000", 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1929999999999993e-06 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.007424+0000", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023375+0000", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015951019 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023416+0000", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0151999999999998e-05 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.023462+0000", 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6675000000000003e-05 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2785 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.862904145000002, 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035965012999999997, 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:38.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2785 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:38.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046717+0000", 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046721+0000", 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046728+0000", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7360000000000004e-06 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.046733+0000", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0479999999999998e-06 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052264+0000", 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055311259999999999 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052344+0000", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0436e-05 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.052737+0000", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039297099999999999 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.054319+0000", 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001581385 2024-09-22T01:14:38.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082507+0000", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028188226 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082629+0000", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012208799999999999 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082682+0000", 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2778000000000002e-05 2024-09-22T01:14:38.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3144 3.e 3:726f12ed:::rbd_data.1124a8bc99f2.0000000000000005:4 [read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.861616965, 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014501714000000001, 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3144 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048004+0000", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048007+0000", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3719999999999999e-06 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048014+0000", 2024-09-22T01:14:38.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9190000000000002e-06 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048019+0000", 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-06 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059199+0000", 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011180099000000001 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.059272+0000", 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2533999999999994e-05 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062506+0000", 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032342019999999998 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3181 3.16 3:69500057:::rbd_object_map.1124ef2748ce:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:38.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.861272782, 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034513777000000002, 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3181 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048348+0000", 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048349+0000", 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048356+0000", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048363+0000", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.048368+0000", 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.464e-06 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062224+0000", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013856187000000001 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062316+0000", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2472999999999994e-05 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.062852+0000", 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053513599999999997 2024-09-22T01:14:38.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.063839+0000", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00098723100000000009 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082753+0000", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018913952000000001 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082812+0000", 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9330999999999998e-05 2024-09-22T01:14:38.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.082862+0000", 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9938999999999997e-05 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:15337 3.e 3:73ab052b:::rbd_data.1124645eb891.0000000000000038:c [sparse-read 720896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.363413252999999, 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017128462000000001, 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15337 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546208+0000", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546210+0000", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8199999999999999e-06 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546215+0000", 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3800000000000002e-06 2024-09-22T01:14:38.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.546219+0000", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0620000000000002e-06 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554742+0000", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085230519999999997 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.554860+0000", 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117883 2024-09-22T01:14:38.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:25.563336+0000", 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084759050000000006 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:21056 3.e 3:70da9618:::rbd_object_map.11247e2569dd:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0099418890000003, 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014372493, 2024-09-22T01:14:38.984 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 21056 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899679+0000", 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899680+0000", 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899683+0000", 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5669999999999999e-06 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899690+0000", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.776e-06 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.899696+0000", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7790000000000001e-06 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911432+0000", 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011736394000000001 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911543+0000", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110684 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.911906+0000", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036337899999999999 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913199+0000", 2024-09-22T01:14:38.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012926000000000001 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.913966+0000", 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00076740200000000004 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914016+0000", 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0039e-05 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.914052+0000", 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5404999999999997e-05 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22363 3.5 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.6279264700000002, 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.025573505, 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:38.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22363 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281695+0000", 2024-09-22T01:14:38.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.281698+0000", 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1769999999999998e-06 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285440+0000", 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037427089999999999 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285474+0000", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3395000000000001e-05 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285490+0000", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6059e-05 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285579+0000", 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8869000000000006e-05 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285746+0000", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167412 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.286076+0000", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000330052 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.297638+0000", 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011562227 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307056+0000", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0094173729999999997 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307155+0000", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8830000000000001e-05 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307268+0000", 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113402 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:22368 3.1e e48/24 3:7c2af90c:::rbd_data.1124dfca90b7.0000000000000006:head v 48'53, mlcod=46'51)", 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.6072464210000001, 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.018959536999999999, 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:38.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302375+0000", 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4000000000000001e-07 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.309944+0000", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0075689340000000003 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.309979+0000", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5136999999999999e-05 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310002+0000", 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2884999999999999e-05 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310151+0000", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014858100000000001 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.310202+0000", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0725e-05 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.321204+0000", 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011002236 2024-09-22T01:14:38.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.321334+0000", 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013039900000000001 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:22367 3.14 e48/28 3:2d346ba6:::rbd_data.1124dfca90b7.0000000000000007:head v 48'118, mlcod=47'116)", 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 7.6071885889999997, 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016597872999999999, 2024-09-22T01:14:39.000 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302432+0000", 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.302434+0000", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2750000000000001e-06 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305003+0000", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025690840000000001 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305030+0000", 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7219000000000001e-05 2024-09-22T01:14:39.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305043+0000", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3253e-05 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305165+0000", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121815 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305213+0000", 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7426000000000003e-05 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.318866+0000", 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013653099 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.319030+0000", 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000164702 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.004 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:39.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:39.048 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:39.046+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:39.111 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:39.110+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:39.172 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:39.172 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:39.172 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:39.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:39.228 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:39.226+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:39.313 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:39.310+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:39.352 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:39.352 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:39.353 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:39.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:39.366 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:39.413 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:39.412+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:39.514 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:39.512+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:39.535 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:14:39.536 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:39.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 44.515394606000001, 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:14:39.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:39.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2201 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:39.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510806563999999, 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046753765000000003, 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2201 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024445+0000", 2024-09-22T01:14:39.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3489999999999999e-06 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024451+0000", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4470000000000003e-06 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024457+0000", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071061+0000", 2024-09-22T01:14:39.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046603415000000002 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071123+0000", 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2509999999999996e-05 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071196+0000", 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2719000000000006e-05 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2202 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510783569000001, 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046876332, 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2202 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024468+0000", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024474+0000", 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2024-09-22T01:14:39.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024480+0000", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071230+0000", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046749717000000003 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071298+0000", 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8266e-05 2024-09-22T01:14:39.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071341+0000", 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3350000000000003e-05 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2203 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:39.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510761345999999, 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046984537999999999, 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2203 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:39.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024488+0000", 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024490+0000", 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024497+0000", 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9970000000000001e-06 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024503+0000", 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.31e-06 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071374+0000", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046871082000000001 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071423+0000", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9172999999999998e-05 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071472+0000", 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8973e-05 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2211 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510738741000001, 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124055999999998, 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2211 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3399999999999999e-07 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024513+0000", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024520+0000", 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7410000000000003e-06 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024533+0000", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2867999999999999e-05 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071509+0000", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046976047 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071569+0000", 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0482999999999997e-05 2024-09-22T01:14:39.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071634+0000", 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4834999999999998e-05 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2212 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510707908000001, 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047215269999999997, 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2212 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3e-07 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024544+0000", 2024-09-22T01:14:39.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0690000000000002e-06 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024551+0000", 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.686e-06 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024557+0000", 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.806e-06 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071661+0000", 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047104241999999998 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071703+0000", 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071756+0000", 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3171999999999998e-05 2024-09-22T01:14:39.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2213 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.566 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510672812999999, 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047299709000000002, 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2213 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024593+0000", 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6223e-05 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024600+0000", 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024607+0000", 2024-09-22T01:14:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080000000000003e-06 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071783+0000", 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047175863999999998 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071822+0000", 2024-09-22T01:14:39.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:39.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071875+0000", 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3801999999999998e-05 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2214 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.510632936, 2024-09-22T01:14:39.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047393246999999999, 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2214 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:39.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026492+0000", 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018767059999999999 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026500+0000", 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026513+0000", 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3422e-05 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026523+0000", 2024-09-22T01:14:39.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071904+0000", 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045380993000000001 2024-09-22T01:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071942+0000", 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572999999999998e-05 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.072009+0000", 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6797000000000004e-05 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2405 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 4915200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506110034999999, 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04671264, 2024-09-22T01:14:39.586 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2405 2024-09-22T01:14:39.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029139+0000", 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029140+0000", 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1209999999999999e-06 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029143+0000", 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2049999999999998e-06 2024-09-22T01:14:39.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029146+0000", 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7800000000000001e-06 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075719+0000", 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046573497999999998 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075787+0000", 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7135e-05 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075851+0000", 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4660999999999996e-05 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2406 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506099091999999, 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046865920999999998, 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2406 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.22e-07 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029151+0000", 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029154+0000", 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.439e-06 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029158+0000", 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075881+0000", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046723187999999999 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075951+0000", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9565000000000002e-05 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076015+0000", 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4931000000000006e-05 2024-09-22T01:14:39.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2407 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506086877000001, 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047019894, 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2407 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3900000000000001e-07 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029163+0000", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2190000000000001e-06 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029167+0000", 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3390000000000001e-06 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029169+0000", 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9249999999999999e-06 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076045+0000", 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046875173999999999 2024-09-22T01:14:39.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076114+0000", 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.902e-05 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076182+0000", 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7978000000000005e-05 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2408 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506075375000002, 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047176774999999997, 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2408 2024-09-22T01:14:39.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.35e-07 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029175+0000", 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029178+0000", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2650000000000001e-06 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029181+0000", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6860000000000002e-06 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076216+0000", 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035615000000003 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076286+0000", 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9680999999999999e-05 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076350+0000", 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4172999999999997e-05 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2409 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506064379000001, 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047331204000000002, 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2409 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-07 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029186+0000", 2024-09-22T01:14:39.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029190+0000", 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1169999999999997e-06 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029194+0000", 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5310000000000003e-06 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076377+0000", 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047182737000000002 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076447+0000", 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0411000000000001e-05 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076515+0000", 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7960999999999995e-05 2024-09-22T01:14:39.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2410 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506032308999998, 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047479322999999997, 2024-09-22T01:14:39.613 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2410 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029217+0000", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029219+0000", 2024-09-22T01:14:39.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.339e-06 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029226+0000", 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6429999999999998e-06 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029231+0000", 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.152e-06 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076544+0000", 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047312694000000002 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076630+0000", 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5753000000000004e-05 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076696+0000", 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6087999999999995e-05 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2411 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.506002986999999, 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047606067000000002, 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2411 2024-09-22T01:14:39.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029249+0000", 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4739999999999999e-06 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029256+0000", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999996e-06 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029260+0000", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076724+0000", 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047463678000000002 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076786+0000", 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000005e-05 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076852+0000", 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5759999999999994e-05 2024-09-22T01:14:39.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2412 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:39.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.505980851, 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047679747000000001, 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2412 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:39.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029270+0000", 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0540000000000002e-06 2024-09-22T01:14:39.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029277+0000", 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029282+0000", 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6890000000000003e-06 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076869+0000", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047587741000000003 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076909+0000", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9504e-05 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076947+0000", 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8578999999999998e-05 2024-09-22T01:14:39.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2413 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.505958753000002, 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047815037999999997, 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2413 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029292+0000", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029299+0000", 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029304+0000", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076974+0000", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047670055000000003 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077036+0000", 2024-09-22T01:14:39.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2019000000000004e-05 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077105+0000", 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9127000000000004e-05 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2414 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.505938022999999, 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048254547000000002, 2024-09-22T01:14:39.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2414 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:39.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029313+0000", 2024-09-22T01:14:39.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.075e-06 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029320+0000", 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029325+0000", 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077133+0000", 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047808311999999999 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077197+0000", 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3912999999999996e-05 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077565+0000", 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830000000000001 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3931 3.11 3:8f505f18:::rbd_data.1124ef2748ce.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.420069827999999, 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048441529999999997, 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3931 2024-09-22T01:14:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:39.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115182+0000", 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115189+0000", 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-09-22T01:14:39.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115194+0000", 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160040+0000", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044845812999999998 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160142+0000", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010177499999999999 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160287+0000", 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014540899999999999 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163457+0000", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031691570000000001 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163508+0000", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1944000000000003e-05 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163581+0000", 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2123e-05 2024-09-22T01:14:39.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163620+0000", 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709000000000002e-05 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.9678308099999997, 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:14:39.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:39.653 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:39.653 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:39.654 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:39.655 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:39.650+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:39.671 INFO:teuthology.orchestra.run.smithi096.stdout:1831 write 0xdc4eeed thru 0xdc56b4a (0x7c5e bytes) 2024-09-22T01:14:39.677 INFO:teuthology.orchestra.run.smithi096.stdout:1833 write 0x3469ea7 thru 0x346f649 (0x57a3 bytes) 2024-09-22T01:14:39.680 INFO:teuthology.orchestra.run.smithi096.stdout:1834 read 0x29c99c0 thru 0x29d8686 (0xecc7 bytes) 2024-09-22T01:14:39.680 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:39.681 INFO:teuthology.orchestra.run.smithi096.stdout:1837 punch from 0x3060186 to 0x306d801, (0xd67b bytes) 2024-09-22T01:14:39.681 INFO:teuthology.orchestra.run.smithi096.stdout:1838 read 0x4107de3 thru 0x4110d29 (0x8f47 bytes) 2024-09-22T01:14:39.681 INFO:teuthology.orchestra.run.smithi096.stdout:1839 read 0xa917780 thru 0xa9261e7 (0xea68 bytes) 2024-09-22T01:14:39.682 INFO:teuthology.orchestra.run.smithi096.stdout:1841 write 0x3d2b39c thru 0x3d386d8 (0xd33d bytes) 2024-09-22T01:14:39.685 INFO:teuthology.orchestra.run.smithi096.stdout:1842 writesame 0xd8a3fff thru 0xd8b1dda data_size 0x6(0xdddc bytes) 2024-09-22T01:14:39.696 INFO:teuthology.orchestra.run.smithi096.stdout:1843 write 0x3e01048 thru 0x3e0ae16 (0x9dcf bytes) 2024-09-22T01:14:39.698 INFO:teuthology.orchestra.run.smithi096.stdout:1845 write 0xbe4e3a thru 0xbeecfe (0x9ec5 bytes) 2024-09-22T01:14:39.719 INFO:teuthology.orchestra.run.smithi096.stdout:1846 trunc from 0xdc56b4b to 0x431a438 2024-09-22T01:14:39.742 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:39.742 INFO:teuthology.orchestra.run.smithi096.stdout:1848 punch from 0x22a9526 to 0x22afd3b, (0x6815 bytes) 2024-09-22T01:14:39.743 INFO:teuthology.orchestra.run.smithi096.stdout:1849 trunc from 0x431a438 to 0x44ceb55 2024-09-22T01:14:39.753 INFO:teuthology.orchestra.run.smithi096.stdout:1851 writesame 0x29d9bed thru 0x29dd16f data_size 0x13(0x3583 bytes) 2024-09-22T01:14:39.754 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:39.750+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:39.756 INFO:teuthology.orchestra.run.smithi096.stdout:1852 read 0x2ef8df6 thru 0x2f02263 (0x946e bytes) 2024-09-22T01:14:39.773 INFO:teuthology.orchestra.run.smithi096.stdout:1854 writesame 0x6c943cd thru 0x6ca2aa4 data_size 0xa6(0xe6d8 bytes) 2024-09-22T01:14:39.775 INFO:teuthology.orchestra.run.smithi096.stdout:1855 punch from 0x3731b0d to 0x373240d, (0x900 bytes) 2024-09-22T01:14:39.775 INFO:teuthology.orchestra.run.smithi096.stdout:1856 write 0x3e326c2 thru 0x3e3bd91 (0x96d0 bytes) 2024-09-22T01:14:39.789 INFO:teuthology.orchestra.run.smithi096.stdout:1857 trunc from 0x6ca2aa5 to 0x140976b 2024-09-22T01:14:39.794 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:39.794 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:39.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24670 3.19 3:99b51832:::rbd_data.11246797a72b.0000000000000004:head [truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:39.792020+0000", 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0036533080000000001, 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0037148540000000001, 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:39.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24670 2024-09-22T01:14:39.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792020+0000", 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:39.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792020+0000", 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792022+0000", 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792058+0000", 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5645000000000002e-05 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792080+0000", 2024-09-22T01:14:39.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2092000000000001e-05 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792091+0000", 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0871e-05 2024-09-22T01:14:39.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:39.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792276+0000", 2024-09-22T01:14:39.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018559699999999999 2024-09-22T01:14:39.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792487+0000", 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000210612 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 1", 2024-09-22T01:14:39.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.792883+0000", 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00039554 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794601+0000", 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0017187459999999999 2024-09-22T01:14:39.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.795670+0000", 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010688430000000001 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.795706+0000", 2024-09-22T01:14:39.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6176000000000001e-05 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.795735+0000", 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8571000000000001e-05 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:24666 3.9 e57/28 3:9569f198:::rbd_data.11246797a72b.0000000000000000:head v 57'145, mlcod=54'143)", 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:39.794361+0000", 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.001311944, 2024-09-22T01:14:39.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001590251, 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794361+0000", 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:39.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794361+0000", 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794364+0000", 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794393+0000", 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8741000000000001e-05 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794421+0000", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8884e-05 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794440+0000", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8862e-05 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794592+0000", 2024-09-22T01:14:39.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015152399999999999 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:39.794642+0000", 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9762999999999998e-05 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 2 2024-09-22T01:14:39.810 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:39.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:39.815 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:39.841 INFO:teuthology.orchestra.run.smithi096.stdout:1859 writesame 0x6df52b0 thru 0x6e02fd2 data_size 0x1(0xdd23 bytes) 2024-09-22T01:14:39.843 INFO:teuthology.orchestra.run.smithi096.stdout:1860 punch from 0x8eb93c to 0x8f139c, (0x5a60 bytes) 2024-09-22T01:14:39.844 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:39.855 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:39.850+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:39.864 INFO:teuthology.orchestra.run.smithi096.stdout:1862 write 0xb71afdc thru 0xb71e866 (0x388b bytes) 2024-09-22T01:14:39.867 INFO:teuthology.orchestra.run.smithi096.stdout:1864 writesame 0x99ba86e thru 0x99c8e26 data_size 0x1(0xe5b9 bytes) 2024-09-22T01:14:39.947 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:39.947 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:39.947 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:39.947 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:39.947 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:39.954 INFO:teuthology.orchestra.run.smithi096.stdout:1865 punch from 0x6a4ce8a to 0x6a5bb96, (0xed0c bytes) 2024-09-22T01:14:39.956 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:39.952+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:39.962 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:39.967 INFO:teuthology.orchestra.run.smithi096.stdout:1866 write 0xb942f7b thru 0xb948cd7 (0x5d5d bytes) 2024-09-22T01:14:39.989 INFO:teuthology.orchestra.run.smithi096.stdout:1868 write 0xe0982f7 thru 0xe0a060e (0x8318 bytes) 2024-09-22T01:14:39.995 INFO:teuthology.orchestra.run.smithi096.stdout:1871 write 0x30e4bcb thru 0x30e8a09 (0x3e3f bytes) 2024-09-22T01:14:39.999 INFO:teuthology.orchestra.run.smithi096.stdout:1872 read 0x8526e10 thru 0x85336b9 (0xc8aa bytes) 2024-09-22T01:14:40.000 INFO:teuthology.orchestra.run.smithi096.stdout:1873 writesame 0xba1a7e0 thru 0xba1c45c data_size 0x3(0x1c7d bytes) 2024-09-22T01:14:40.005 INFO:teuthology.orchestra.run.smithi096.stdout:1874 write 0x45f6286 thru 0x45f961c (0x3397 bytes) 2024-09-22T01:14:40.009 INFO:teuthology.orchestra.run.smithi096.stdout:1875 punch from 0x408a8df to 0x4091ab6, (0x71d7 bytes) 2024-09-22T01:14:40.009 INFO:teuthology.orchestra.run.smithi096.stdout:1876 writesame 0xc5cf615 thru 0xc5d992b data_size 0x3(0xa317 bytes) 2024-09-22T01:14:40.032 INFO:teuthology.orchestra.run.smithi096.stdout:1877 clone 17 order 23 su 32768 sc 3 2024-09-22T01:14:40.056 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:40.056+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:40.130 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.777005222, 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:40.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:40.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776939056, 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:40.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776879019000001, 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:40.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776843957000001, 2024-09-22T01:14:40.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:40.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776813687000001, 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776784114, 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:40.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:40.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:40.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:40.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776730023000001, 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:40.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776674179, 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:40.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776616289, 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:40.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:40.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776554556000001, 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:40.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776459093, 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.77639727, 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.182 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.77630613, 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:40.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.186 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776244223999999, 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.187 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776183737, 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:40.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:40.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776126267, 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:40.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:40.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:40.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.199 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776068987, 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:40.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:40.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.776009462999999, 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:40.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:40.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:40.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.775956579000001, 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.766844017, 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.218 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:40.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:40.219 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:40.218+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:40.320 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:40.316+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:40.386 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:40.387 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:40.387 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:40.387 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:40.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:40.420 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:40.418+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:40.521 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:40.520+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:40.575 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:40.575 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:40.575 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:40.575 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:40.576 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:40.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:40.632 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:40.632+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:40.723 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:40.720+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:40.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20393 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418464014, 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042022144999999997, 2024-09-22T01:14:40.760 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20393 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340295+0000", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5799999999999998e-07 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340299+0000", 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1540000000000004e-06 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340313+0000", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3854e-05 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340327+0000", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4151000000000001e-05 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381789+0000", 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041462496000000001 2024-09-22T01:14:40.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381873+0000", 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4049999999999997e-05 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382316+0000", 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044248200000000001 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20394 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418415914000001, 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042595531999999998, 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20394 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340343+0000", 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.95e-07 2024-09-22T01:14:40.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340348+0000", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340362+0000", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4744e-05 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340373+0000", 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0411000000000001e-05 2024-09-22T01:14:40.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382394+0000", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042020841000000003 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382511+0000", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117518 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382937+0000", 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042600899999999998 2024-09-22T01:14:40.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20395 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3538944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418370426999999, 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043113203000000003, 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.769 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20395 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340388+0000", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8100000000000001e-07 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340393+0000", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7679999999999999e-06 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340408+0000", 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4924e-05 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340420+0000", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2476e-05 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382978+0000", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042557375000000001 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383074+0000", 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5839000000000002e-05 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383500+0000", 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042684 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20396 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.41832284, 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043547334, 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20396 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:40.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340436+0000", 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4300000000000001e-07 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340456+0000", 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0012000000000001e-05 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340472+0000", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6302000000000001e-05 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340482+0000", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0453e-05 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383550+0000", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043067831000000001 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:40.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383628+0000", 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383982+0000", 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035378500000000002 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20397 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418258779, 2024-09-22T01:14:40.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043522706000000001, 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20397 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:40.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340500+0000", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340505+0000", 2024-09-22T01:14:40.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2100000000000001e-06 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340520+0000", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4865000000000001e-05 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340544+0000", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3708000000000001e-05 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384013+0000", 2024-09-22T01:14:40.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043469926999999998 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384022+0000", 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0639999999999994e-06 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20398 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418197965999999, 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043503679000000003, 2024-09-22T01:14:40.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20398 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340561+0000", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340566+0000", 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1619999999999999e-06 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340580+0000", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4633e-05 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340592+0000", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1134e-05 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384056+0000", 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043464964000000002 2024-09-22T01:14:40.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384063+0000", 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20399 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418148154000001, 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043483817000000001, 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20399 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340611+0000", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.28e-06 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340616+0000", 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0849999999999996e-06 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340631+0000", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5384999999999999e-05 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340642+0000", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0644e-05 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384087+0000", 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043445399000000003 2024-09-22T01:14:40.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384093+0000", 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20400 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.41810143, 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043466537, 2024-09-22T01:14:40.789 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20400 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340657+0000", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340662+0000", 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1279999999999999e-06 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340677+0000", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5279999999999999e-05 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340688+0000", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0689999999999999e-05 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384117+0000", 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043428384 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384123+0000", 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1700000000000002e-06 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20401 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.418054723999999, 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043449197000000002, 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.793 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20401 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340704+0000", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340718+0000", 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3804e-05 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340733+0000", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5013e-05 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340744+0000", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1207e-05 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384145+0000", 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043401020999999998 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384152+0000", 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.063e-06 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20402 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417997832999999, 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043424268000000002, 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20402 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340761+0000", 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1699999999999997e-07 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340766+0000", 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.215e-06 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340781+0000", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5065000000000001e-05 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340792+0000", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0556e-05 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384177+0000", 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043385328000000001 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384184+0000", 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.187e-06 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20403 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417951372999999, 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043422686000000002, 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.801 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20403 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340807+0000", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.47e-07 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340812+0000", 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340828+0000", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5553000000000002e-05 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340837+0000", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3870000000000001e-06 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384220+0000", 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043383163000000002 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384229+0000", 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20404 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417904765999999, 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043441768999999998, 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20404 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340854+0000", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7899999999999997e-07 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340859+0000", 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9969999999999998e-06 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340874+0000", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5404e-05 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340885+0000", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0916e-05 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384283+0000", 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043397463999999997 2024-09-22T01:14:40.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384295+0000", 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2109e-05 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20405 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417856180999999, 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043420168000000002, 2024-09-22T01:14:40.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20405 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340903+0000", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.372e-06 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340908+0000", 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1560000000000003e-06 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340924+0000", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5829999999999999e-05 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340935+0000", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0951000000000001e-05 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384316+0000", 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043380836999999998 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384322+0000", 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0220000000000001e-06 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20406 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.41780741, 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043418238999999997, 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20406 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340952+0000", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3030000000000001e-06 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340956+0000", 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7790000000000003e-06 2024-09-22T01:14:40.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340971+0000", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5196e-05 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340997+0000", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5863999999999998e-05 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384356+0000", 2024-09-22T01:14:40.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358529999999999 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384368+0000", 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2567e-05 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20407 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417744446, 2024-09-22T01:14:40.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043399115000000002, 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20407 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341014+0000", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341031+0000", 2024-09-22T01:14:40.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6769000000000002e-05 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341047+0000", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341058+0000", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1079e-05 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384406+0000", 2024-09-22T01:14:40.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043347572000000001 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384412+0000", 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2439999999999998e-06 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20408 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417681980999999, 2024-09-22T01:14:40.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043369389000000001, 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20408 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:40.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341077+0000", 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341082+0000", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8879999999999996e-06 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341099+0000", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7269999999999999e-05 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341110+0000", 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0919e-05 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384438+0000", 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043328158999999998 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384445+0000", 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.951e-06 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20409 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.417633469, 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043398471000000001, 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20409 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341125+0000", 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5799999999999998e-07 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341130+0000", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341145+0000", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5386000000000001e-05 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341157+0000", 2024-09-22T01:14:40.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1423999999999999e-05 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384515+0000", 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358514000000001 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384523+0000", 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3950000000000003e-06 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20561 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.416112615999999, 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041926151000000002, 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20561 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342648+0000", 2024-09-22T01:14:40.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3289999999999998e-06 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342653+0000", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342669+0000", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6081000000000001e-05 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342681+0000", 2024-09-22T01:14:40.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1382999999999999e-05 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384562+0000", 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041881599999999998 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384571+0000", 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9730000000000003e-06 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20562 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.416044852000001, 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041931077999999997, 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20562 2024-09-22T01:14:40.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342714+0000", 2024-09-22T01:14:40.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000004e-07 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342719+0000", 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2549999999999997e-06 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342736+0000", 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7144999999999999e-05 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342747+0000", 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1080999999999999e-05 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384631+0000", 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041884059000000001 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384644+0000", 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2619999999999999e-05 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20563 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.4159945, 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041909158000000002, 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:40.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20563 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:40.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342764+0000", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.068e-06 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342779+0000", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5302999999999999e-05 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342796+0000", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6798000000000001e-05 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:40.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342807+0000", 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1024e-05 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384665+0000", 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041858039999999999 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384672+0000", 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:40.842 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:40.842 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:40.843 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:40.843 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:40.842+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:40.892 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:40.892 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:40.908 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:40.908 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:40.924 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:40.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:40.941 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:40.941 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:40.943 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:40.942+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:40.959 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:40.959 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:40.978 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:40.978 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:40.998 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:40.999 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:41.015 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.015 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.029 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:41.072 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:41.072+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:41.145 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:41.142+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:41.146 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.147 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.160 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:41.222 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:41.222 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:41.252 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:41.252+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:41.327 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.327 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:41.347 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:41.342+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:41.447 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:41.444+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:41.505 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.506 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:41.548 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:41.546+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:41.649 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:41.646+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:41.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:14:41.688 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.688 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.704 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:41.749 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:41.746+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:41.759 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone16 from 0xe0a060f (overlap 0x41636a) to 0x273ba6 2024-09-22T01:14:41.835 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.835 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.849 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:41.850 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:41.846+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:41.951 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:41.948+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:41.979 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:41.979 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:41.993 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-22T01:14:41.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:42.049 INFO:teuthology.orchestra.run.smithi096.stdout:1878 read 0x461d581 thru 0x46209e2 (0x3462 bytes) 2024-09-22T01:14:42.051 INFO:teuthology.orchestra.run.smithi096.stdout:1879 read 0xb233f65 thru 0xb2353cb (0x1467 bytes) 2024-09-22T01:14:42.052 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:42.052+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:42.053 INFO:teuthology.orchestra.run.smithi096.stdout:1880 writesame 0xb95743a thru 0xb960872 data_size 0x1(0x9439 bytes) 2024-09-22T01:14:42.130 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.130 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.138 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:42.139 INFO:teuthology.orchestra.run.smithi096.stdout:1882 read 0x79645b6 thru 0x796d970 (0x93bb bytes) 2024-09-22T01:14:42.139 INFO:teuthology.orchestra.run.smithi096.stdout:1883 write 0xa877e66 thru 0xa8817c0 (0x995b bytes) 2024-09-22T01:14:42.145 INFO:teuthology.orchestra.run.smithi096.stdout:1884 read 0x2546580 thru 0x2546b07 (0x588 bytes) 2024-09-22T01:14:42.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:42.147 INFO:teuthology.orchestra.run.smithi096.stdout:1885 writesame 0x20a0642 thru 0x20a84b4 data_size 0x1(0x7e73 bytes) 2024-09-22T01:14:42.153 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:42.152+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:1886 read 0x4721501 thru 0x472193d (0x43d bytes) 2024-09-22T01:14:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:1887 punch from 0x4ce070a to 0x4ce9070, (0x8966 bytes) 2024-09-22T01:14:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:42.203 INFO:teuthology.orchestra.run.smithi096.stdout:1891 writesame 0x7ea913a thru 0x7eb81d8 data_size 0x1(0xf09f bytes) 2024-09-22T01:14:42.254 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:42.252+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:42.274 INFO:teuthology.orchestra.run.smithi096.stdout:1892 writesame 0x3eafce8 thru 0x3eb674c data_size 0x7(0x6a65 bytes) 2024-09-22T01:14:42.303 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.304 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:42.329 INFO:teuthology.orchestra.run.smithi096.stdout:1893 punch from 0x3e8a5be to 0x3e954f2, (0xaf34 bytes) 2024-09-22T01:14:42.329 INFO:teuthology.orchestra.run.smithi096.stdout:1894 read 0x1950a5e thru 0x196012a (0xf6cd bytes) 2024-09-22T01:14:42.330 INFO:teuthology.orchestra.run.smithi096.stdout:1895 punch from 0x16e61f1 to 0x16f32db, (0xd0ea bytes) 2024-09-22T01:14:42.330 INFO:teuthology.orchestra.run.smithi096.stdout:1896 writesame 0xbe56d0 thru 0xbf32f2 data_size 0xf(0xdc23 bytes) 2024-09-22T01:14:42.358 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:42.352+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:42.453 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.453 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.456 INFO:teuthology.orchestra.run.smithi096.stdout:1897 trunc from 0xe0a060f to 0x4e29a6d 2024-09-22T01:14:42.456 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:42.456+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:42.469 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:42.546 INFO:teuthology.orchestra.run.smithi096.stdout:1898 punch from 0x29ebe02 to 0x29f033f, (0x453d bytes) 2024-09-22T01:14:42.546 INFO:teuthology.orchestra.run.smithi096.stdout:1899 punch from 0x41dc1c0 to 0x41e5bca, (0x9a0a bytes) 2024-09-22T01:14:42.546 INFO:teuthology.orchestra.run.smithi096.stdout:1900 write 0x1638de6 thru 0x163d6f0 (0x490b bytes) 2024-09-22T01:14:42.548 INFO:teuthology.orchestra.run.smithi096.stdout:1901 read 0x3e08ed4 thru 0x3e143d2 (0xb4ff bytes) 2024-09-22T01:14:42.549 INFO:teuthology.orchestra.run.smithi096.stdout:1903 read 0x39f6b26 thru 0x3a00919 (0x9df4 bytes) 2024-09-22T01:14:42.550 INFO:teuthology.orchestra.run.smithi096.stdout:1904 read 0x12cce41 thru 0x12d0c79 (0x3e39 bytes) 2024-09-22T01:14:42.552 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:42.552 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:42.552 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-22T01:14:42.553 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:14:42.553 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:14:42.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-22T01:14:42.560 INFO:teuthology.orchestra.run.smithi096.stdout:1906 writesame 0x5d244ce thru 0x5d2f83d data_size 0x12(0xb370 bytes) 2024-09-22T01:14:42.561 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:42.558+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:42.561 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:14:42.564 INFO:teuthology.orchestra.run.smithi096.stdout:1907 read 0x530ae6f thru 0x53114d3 (0x6665 bytes) 2024-09-22T01:14:42.568 INFO:teuthology.orchestra.run.smithi096.stdout:1908 read 0x37440ce thru 0x3749bf6 (0x5b29 bytes) 2024-09-22T01:14:42.602 INFO:teuthology.orchestra.run.smithi096.stdout:1910 writesame 0xe08ae91 thru 0xe0960c6 data_size 0x1(0xb236 bytes) 2024-09-22T01:14:42.602 INFO:teuthology.orchestra.run.smithi096.stdout:1911 read 0xa163a96 thru 0xa16c94a (0x8eb5 bytes) 2024-09-22T01:14:42.602 INFO:teuthology.orchestra.run.smithi096.stdout:1912 punch from 0xc4c951c to 0xc4d0b40, (0x7624 bytes) 2024-09-22T01:14:42.603 INFO:teuthology.orchestra.run.smithi096.stdout:1913 read 0x45e20e9 thru 0x45ec4ff (0xa417 bytes) 2024-09-22T01:14:42.603 INFO:teuthology.orchestra.run.smithi096.stdout:1914 read 0x9088788 thru 0x9097487 (0xed00 bytes) 2024-09-22T01:14:42.603 INFO:teuthology.orchestra.run.smithi096.stdout:1916 write 0xa55f7d4 thru 0xa563621 (0x3e4e bytes) 2024-09-22T01:14:42.611 INFO:teuthology.orchestra.run.smithi096.stdout:1917 write 0x42e8c98 thru 0x42ed045 (0x43ae bytes) 2024-09-22T01:14:42.616 INFO:teuthology.orchestra.run.smithi096.stdout:1918 punch from 0x43c0470 to 0x43c846a, (0x7ffa bytes) 2024-09-22T01:14:42.616 INFO:teuthology.orchestra.run.smithi096.stdout:1921 clone 18 order 19 su 262144 sc 15 2024-09-22T01:14:42.645 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.645 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.658 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:42.654+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:42.659 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:42.675 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:42.676 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:14:42.676 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:42.759 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:42.756+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:42.791 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.792 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.805 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:42.860 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:42.860+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:42.927 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:42.927 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:42.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:42.961 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:42.960+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:43.062 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:43.060+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:43.064 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:43.065 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:43.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:43.162 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:43.160+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:43.204 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:43.204 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:43.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:43.243 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:43.263 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:43.262+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:43.342 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:43.342 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:43.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:43.364 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:43.360+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:43.403 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:43.403 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:43.420 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:43.420 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:43.436 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:43.437 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:43.453 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:43.453 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:43.464 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:43.464+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:43.473 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:43.474 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:43.475 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:43.475 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:43.476 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:43.476 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:43.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:43.495 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:43.495 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:43.515 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:43.516 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:43.537 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:43.537 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:43.603 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:43.602+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:43.625 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.605148086, 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:14:43.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:14:43.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:14:43.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2201 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7667712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600560044000002, 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046753765000000003, 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2201 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024442+0000", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024445+0000", 2024-09-22T01:14:43.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3489999999999999e-06 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024451+0000", 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4470000000000003e-06 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024457+0000", 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9519999999999999e-06 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071061+0000", 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046603415000000002 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071123+0000", 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2509999999999996e-05 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071196+0000", 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2719000000000006e-05 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2202 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600537049, 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046876332, 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2202 2024-09-22T01:14:43.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024465+0000", 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024468+0000", 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024474+0000", 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3620000000000002e-06 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024480+0000", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071230+0000", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046749717000000003 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071298+0000", 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8266e-05 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071341+0000", 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3350000000000003e-05 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2203 3.9 3:963cf37a:::rbd_data.1124a8bc99f2.000000000000000c:4 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600514826000001, 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046984537999999999, 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2203 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024487+0000", 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024488+0000", 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024490+0000", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5940000000000001e-06 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024497+0000", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9970000000000001e-06 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024503+0000", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.31e-06 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071374+0000", 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046871082000000001 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071423+0000", 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9172999999999998e-05 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071472+0000", 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8973e-05 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2211 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 196608~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600492221, 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047124055999999998, 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2211 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024510+0000", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3399999999999999e-07 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024513+0000", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4480000000000001e-06 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024520+0000", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7410000000000003e-06 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024533+0000", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2867999999999999e-05 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071509+0000", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046976047 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071569+0000", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0482999999999997e-05 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071634+0000", 2024-09-22T01:14:43.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4834999999999998e-05 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2212 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 524288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600461387999999, 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047215269999999997, 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2212 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024541+0000", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3e-07 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024544+0000", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0690000000000002e-06 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024551+0000", 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.686e-06 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024557+0000", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.806e-06 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071661+0000", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047104241999999998 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071703+0000", 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1665000000000002e-05 2024-09-22T01:14:43.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071756+0000", 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3171999999999998e-05 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2213 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600426293000002, 2024-09-22T01:14:43.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047299709000000002, 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2213 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024576+0000", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024593+0000", 2024-09-22T01:14:43.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6223e-05 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024600+0000", 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7770000000000002e-06 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024607+0000", 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4080000000000003e-06 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071783+0000", 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047175863999999998 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071822+0000", 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9042000000000003e-05 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071875+0000", 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3801999999999998e-05 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2214 3.9 3:96e87a2a:::rbd_data.1124a8bc99f2.000000000000000e:4 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.600386415999999, 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047393246999999999, 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2214 2024-09-22T01:14:43.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.024616+0000", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026492+0000", 2024-09-22T01:14:43.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018767059999999999 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026500+0000", 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6199999999999999e-06 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026513+0000", 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3422e-05 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.026523+0000", 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071904+0000", 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045380993000000001 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.071942+0000", 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8572999999999998e-05 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.072009+0000", 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6797000000000004e-05 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2406 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595852571999998, 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046865920999999998, 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2406 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029150+0000", 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.22e-07 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029151+0000", 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029154+0000", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.439e-06 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029158+0000", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4790000000000001e-06 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075881+0000", 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046723187999999999 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.075951+0000", 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9565000000000002e-05 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076015+0000", 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4931000000000006e-05 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2407 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595840357, 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047019894, 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2407 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029162+0000", 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3900000000000001e-07 2024-09-22T01:14:43.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029163+0000", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2190000000000001e-06 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029167+0000", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3390000000000001e-06 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029169+0000", 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9249999999999999e-06 2024-09-22T01:14:43.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076045+0000", 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046875173999999999 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076114+0000", 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.902e-05 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076182+0000", 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7978000000000005e-05 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2408 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6127616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595828855000001, 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047176774999999997, 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2408 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029173+0000", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.35e-07 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029175+0000", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029178+0000", 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2650000000000001e-06 2024-09-22T01:14:43.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029181+0000", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6860000000000002e-06 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076216+0000", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035615000000003 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076286+0000", 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9680999999999999e-05 2024-09-22T01:14:43.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076350+0000", 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4172999999999997e-05 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2409 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6356992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595817859, 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047331204000000002, 2024-09-22T01:14:43.676 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2409 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029184+0000", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4499999999999998e-07 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029186+0000", 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029190+0000", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1169999999999997e-06 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029194+0000", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5310000000000003e-06 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076377+0000", 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047182737000000002 2024-09-22T01:14:43.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076447+0000", 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0411000000000001e-05 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076515+0000", 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7960999999999995e-05 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2410 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 6684672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595785789000001, 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047479322999999997, 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2410 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029216+0000", 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029217+0000", 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029219+0000", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.339e-06 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029226+0000", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6429999999999998e-06 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029231+0000", 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.152e-06 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076544+0000", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047312694000000002 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076630+0000", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5753000000000004e-05 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076696+0000", 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6087999999999995e-05 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2411 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595756467000001, 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047606067000000002, 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2411 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029246+0000", 2024-09-22T01:14:43.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029249+0000", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4739999999999999e-06 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029256+0000", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999996e-06 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029260+0000", 2024-09-22T01:14:43.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7910000000000004e-06 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076724+0000", 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047463678000000002 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076786+0000", 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000005e-05 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076852+0000", 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5759999999999994e-05 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2412 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595734330999999, 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047679747000000001, 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2412 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029268+0000", 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000005e-07 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029270+0000", 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0540000000000002e-06 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029277+0000", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6789999999999999e-06 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029282+0000", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6890000000000003e-06 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076869+0000", 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047587741000000003 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076909+0000", 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9504e-05 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076947+0000", 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8578999999999998e-05 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2413 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595712233, 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047815037999999997, 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2413 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029290+0000", 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:14:43.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029292+0000", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0499999999999999e-06 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029299+0000", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4359999999999999e-06 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029304+0000", 2024-09-22T01:14:43.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9640000000000002e-06 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.076974+0000", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047670055000000003 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077036+0000", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2019000000000004e-05 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077105+0000", 2024-09-22T01:14:43.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9127000000000004e-05 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:2414 3.9 3:96132823:::rbd_data.1124a8bc99f2.0000000000000009:4 [read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.595691503000001, 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048254547000000002, 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2414 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029311+0000", 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5599999999999995e-07 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029313+0000", 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.075e-06 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029320+0000", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4030000000000004e-06 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.029325+0000", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9880000000000004e-06 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077133+0000", 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047808311999999999 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077197+0000", 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3912999999999996e-05 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.077565+0000", 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036830000000000001 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:3931 3.11 3:8f505f18:::rbd_data.1124ef2748ce.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1703936~11010048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-22T01:14:43.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.509823308000001, 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048441529999999997, 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3931 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115179+0000", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115182+0000", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0769999999999999e-06 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115189+0000", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9269999999999999e-06 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.115194+0000", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160040+0000", 2024-09-22T01:14:43.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044845812999999998 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160142+0000", 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010177499999999999 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.160287+0000", 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014540899999999999 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:14:43.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163457+0000", 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0031691570000000001 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163508+0000", 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1944000000000003e-05 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163581+0000", 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2123e-05 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:12.163620+0000", 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709000000000002e-05 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.0575842899999994, 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:14:43.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:14:43.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.0462736250000004, 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:14:43.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:14:43.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:14:43.718 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:43.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:43.719 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:43.716+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:43.817 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone17 from 0xe0960c7 (overlap 0x4e29a6d) to 0x2714d36 2024-09-22T01:14:43.820 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:43.818+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:43.861 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:43.861 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:43.861 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:43.861 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:43.875 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:43.921 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:43.918+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:43.998 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:43.998 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:43.998 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:43.998 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:43.999 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:44.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:44.022 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:44.018+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:44.123 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:44.120+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:44.137 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:44.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783304637000001, 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.138 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783238471000001, 2024-09-22T01:14:44.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:44.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:44.146 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783178434, 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:44.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:44.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:44.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783143372, 2024-09-22T01:14:44.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:44.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:44.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783113102, 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:44.160 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783083529000001, 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:44.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.783029438, 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:44.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:44.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782973594, 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782915704000001, 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782853971, 2024-09-22T01:14:44.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:44.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.181 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782758508000001, 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:44.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782696684999999, 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782605544999999, 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.190 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782543639, 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:44.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:44.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.199 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782483151999999, 2024-09-22T01:14:44.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:44.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782425682, 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:44.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:44.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:44.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:44.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:44.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:44.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782368401999999, 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:44.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.222 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782308878, 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.226 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.782255994, 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.227 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:44.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:44.231 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.773143431999999, 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:44.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:44.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.237 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:44.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:44.239 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:44.236+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:44.339 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:44.336+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:44.352 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-22T01:14:44.366 INFO:teuthology.orchestra.run.smithi096.stdout:1922 writesame 0x228a144 thru 0x2293821 data_size 0x29(0x96de bytes) 2024-09-22T01:14:44.382 INFO:teuthology.orchestra.run.smithi096.stdout:1923 writesame 0x5c2b14b thru 0x5c38eab data_size 0x9(0xdd61 bytes) 2024-09-22T01:14:44.396 INFO:teuthology.orchestra.run.smithi096.stdout:1924 writesame 0x59b4574 thru 0x59b9aec data_size 0x1(0x5579 bytes) 2024-09-22T01:14:44.415 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:44.416 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:44.428 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.429 INFO:teuthology.orchestra.run.smithi096.stdout:1926 trunc from 0xe0960c7 to 0xe4309ab 2024-09-22T01:14:44.430 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:44.440 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:44.438+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:44.441 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.441 INFO:teuthology.orchestra.run.smithi096.stdout:1929 writesame 0xb7a4cf7 thru 0xb7a8159 data_size 0x1(0x3463 bytes) 2024-09-22T01:14:44.465 INFO:teuthology.orchestra.run.smithi096.stdout:1930 punch from 0x94f7eb5 to 0x950001d, (0x8168 bytes) 2024-09-22T01:14:44.465 INFO:teuthology.orchestra.run.smithi096.stdout:1932 writesame 0xac1359 thru 0xad00ca data_size 0x63(0xed72 bytes) 2024-09-22T01:14:44.478 INFO:teuthology.orchestra.run.smithi096.stdout:1933 punch from 0x5f3ea8c to 0x5f4a8c9, (0xbe3d bytes) 2024-09-22T01:14:44.478 INFO:teuthology.orchestra.run.smithi096.stdout:1934 read 0xd6beda1 thru 0xd6c48a9 (0x5b09 bytes) 2024-09-22T01:14:44.478 INFO:teuthology.orchestra.run.smithi096.stdout:1935 punch from 0x25afa3f to 0x25b83b7, (0x8978 bytes) 2024-09-22T01:14:44.478 INFO:teuthology.orchestra.run.smithi096.stdout:1936 write 0xd93c634 thru 0xd93e224 (0x1bf1 bytes) 2024-09-22T01:14:44.486 INFO:teuthology.orchestra.run.smithi096.stdout:1937 write 0x3bf9e16 thru 0x3c026f4 (0x88df bytes) 2024-09-22T01:14:44.496 INFO:teuthology.orchestra.run.smithi096.stdout:1938 write 0x92cd83b thru 0x92d7d22 (0xa4e8 bytes) 2024-09-22T01:14:44.503 INFO:teuthology.orchestra.run.smithi096.stdout:1939 writesame 0xa3294e6 thru 0xa330cb9 data_size 0x2(0x77d4 bytes) 2024-09-22T01:14:44.542 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:44.538+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:44.566 INFO:teuthology.orchestra.run.smithi096.stdout:1940 trunc from 0xe4309ab to 0xa6c22d 2024-09-22T01:14:44.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:14:44.613 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:44.613 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:44.613 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:44.613 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:44.614 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:44.631 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:44.638 INFO:teuthology.orchestra.run.smithi096.stdout:1941 punch from 0x8c63f to 0x9588a, (0x924b bytes) 2024-09-22T01:14:44.649 INFO:teuthology.orchestra.run.smithi096.stdout:1943 trunc from 0xa6c22d to 0x654b140 2024-09-22T01:14:44.656 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.663 INFO:teuthology.orchestra.run.smithi096.stdout:1945 write 0x67279dc thru 0x6728e5b (0x1480 bytes) 2024-09-22T01:14:44.674 INFO:teuthology.orchestra.run.smithi096.stdout:1948 trunc from 0x6728e5c to 0x2804305 2024-09-22T01:14:44.676 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:44.676+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:44.716 INFO:teuthology.orchestra.run.smithi096.stdout:1949 write 0x8857ad3 thru 0x885b88a (0x3db8 bytes) 2024-09-22T01:14:44.722 INFO:teuthology.orchestra.run.smithi096.stdout:1950 read 0x1298859 thru 0x12a3c2e (0xb3d6 bytes) 2024-09-22T01:14:44.722 INFO:teuthology.orchestra.run.smithi096.stdout:1952 read 0x652ebdb thru 0x6534c0f (0x6035 bytes) 2024-09-22T01:14:44.723 INFO:teuthology.orchestra.run.smithi096.stdout:1953 write 0x6ab5043 thru 0x6abebb0 (0x9b6e bytes) 2024-09-22T01:14:44.728 INFO:teuthology.orchestra.run.smithi096.stdout:1954 read 0x48fe9f7 thru 0x48ff5f5 (0xbff bytes) 2024-09-22T01:14:44.728 INFO:teuthology.orchestra.run.smithi096.stdout:1956 read 0x82e1694 thru 0x82ecd39 (0xb6a6 bytes) 2024-09-22T01:14:44.730 INFO:teuthology.orchestra.run.smithi096.stdout:1957 trunc from 0x885b88b to 0x9b60761 2024-09-22T01:14:44.738 INFO:teuthology.orchestra.run.smithi096.stdout:1959 writesame 0x9ac1b1a thru 0x9ac960a data_size 0x27(0x7af1 bytes) 2024-09-22T01:14:44.743 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:44.740+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:44.744 INFO:teuthology.orchestra.run.smithi096.stdout:1960 punch from 0x9a20ad1 to 0x9a22423, (0x1952 bytes) 2024-09-22T01:14:44.744 INFO:teuthology.orchestra.run.smithi096.stdout:1963 write 0x8fcff69 thru 0x8fd4c0f (0x4ca7 bytes) 2024-09-22T01:14:44.748 INFO:teuthology.orchestra.run.smithi096.stdout:1964 punch from 0x534a787 to 0x5353411, (0x8c8a bytes) 2024-09-22T01:14:44.756 INFO:teuthology.orchestra.run.smithi096.stdout:1966 write 0xa4b6b27 thru 0xa4c192b (0xae05 bytes) 2024-09-22T01:14:44.761 INFO:teuthology.orchestra.run.smithi096.stdout:1967 write 0x6748cb6 thru 0x6755933 (0xcc7e bytes) 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20393 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424796505, 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042022144999999997, 2024-09-22T01:14:44.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20393 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:44.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340295+0000", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5799999999999998e-07 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340299+0000", 2024-09-22T01:14:44.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1540000000000004e-06 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340313+0000", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3854e-05 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340327+0000", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4151000000000001e-05 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381789+0000", 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041462496000000001 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381873+0000", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4049999999999997e-05 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382316+0000", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044248200000000001 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20394 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:44.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424748405000001, 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042595531999999998, 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20394 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:44.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340343+0000", 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.95e-07 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340348+0000", 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340362+0000", 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4744e-05 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340373+0000", 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0411000000000001e-05 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382394+0000", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042020841000000003 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382511+0000", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117518 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382937+0000", 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042600899999999998 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20395 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3538944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424702917999999, 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043113203000000003, 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20395 2024-09-22T01:14:44.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340388+0000", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8100000000000001e-07 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340393+0000", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7679999999999999e-06 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340408+0000", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4924e-05 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340420+0000", 2024-09-22T01:14:44.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2476e-05 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382978+0000", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042557375000000001 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383074+0000", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5839000000000002e-05 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383500+0000", 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042684 2024-09-22T01:14:44.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20396 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424655331, 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043547334, 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20396 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340436+0000", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4300000000000001e-07 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340456+0000", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0012000000000001e-05 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340472+0000", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6302000000000001e-05 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340482+0000", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0453e-05 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383550+0000", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043067831000000001 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383628+0000", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383982+0000", 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035378500000000002 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20397 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424591270000001, 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043522706000000001, 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20397 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340500+0000", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340505+0000", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2100000000000001e-06 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340520+0000", 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4865000000000001e-05 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340544+0000", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3708000000000001e-05 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384013+0000", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043469926999999998 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384022+0000", 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0639999999999994e-06 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20398 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424530456999999, 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043503679000000003, 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20398 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.786 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340561+0000", 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340566+0000", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1619999999999999e-06 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340580+0000", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4633e-05 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340592+0000", 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1134e-05 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384056+0000", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043464964000000002 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384063+0000", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20399 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:44.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424480644999999, 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043483817000000001, 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20399 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340611+0000", 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.28e-06 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340616+0000", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0849999999999996e-06 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340631+0000", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5384999999999999e-05 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340642+0000", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0644e-05 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384087+0000", 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043445399000000003 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384093+0000", 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20400 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424433921, 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043466537, 2024-09-22T01:14:44.793 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20400 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340657+0000", 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340662+0000", 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1279999999999999e-06 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340677+0000", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5279999999999999e-05 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340688+0000", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0689999999999999e-05 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384117+0000", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043428384 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384123+0000", 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1700000000000002e-06 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20401 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424387214999999, 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043449197000000002, 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20401 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340704+0000", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340718+0000", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3804e-05 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340733+0000", 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5013e-05 2024-09-22T01:14:44.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340744+0000", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1207e-05 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384145+0000", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043401020999999998 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384152+0000", 2024-09-22T01:14:44.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.063e-06 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20402 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424330324, 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043424268000000002, 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20402 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340761+0000", 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1699999999999997e-07 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340766+0000", 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.215e-06 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340781+0000", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5065000000000001e-05 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340792+0000", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0556e-05 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384177+0000", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043385328000000001 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384184+0000", 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.187e-06 2024-09-22T01:14:44.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20403 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424283864, 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043422686000000002, 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20403 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340807+0000", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.47e-07 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340812+0000", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340828+0000", 2024-09-22T01:14:44.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5553000000000002e-05 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340837+0000", 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3870000000000001e-06 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384220+0000", 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043383163000000002 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384229+0000", 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20404 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424237257, 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043441768999999998, 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20404 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340854+0000", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7899999999999997e-07 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340859+0000", 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9969999999999998e-06 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340874+0000", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5404e-05 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340885+0000", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0916e-05 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384283+0000", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043397463999999997 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384295+0000", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2109e-05 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20405 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424188672, 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043420168000000002, 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20405 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:44.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340903+0000", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.372e-06 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340908+0000", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1560000000000003e-06 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340924+0000", 2024-09-22T01:14:44.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5829999999999999e-05 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340935+0000", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0951000000000001e-05 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384316+0000", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043380836999999998 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384322+0000", 2024-09-22T01:14:44.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0220000000000001e-06 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20406 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424139901, 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043418238999999997, 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20406 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340952+0000", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3030000000000001e-06 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340956+0000", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7790000000000003e-06 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340971+0000", 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5196e-05 2024-09-22T01:14:44.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340997+0000", 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5863999999999998e-05 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384356+0000", 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358529999999999 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384368+0000", 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2567e-05 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20407 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424076937000001, 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043399115000000002, 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20407 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:44.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341014+0000", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341031+0000", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6769000000000002e-05 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341047+0000", 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:44.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341058+0000", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1079e-05 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384406+0000", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043347572000000001 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384412+0000", 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2439999999999998e-06 2024-09-22T01:14:44.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20408 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.424014472, 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043369389000000001, 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.825 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20408 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:44.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341077+0000", 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341082+0000", 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8879999999999996e-06 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341099+0000", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7269999999999999e-05 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341110+0000", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0919e-05 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384438+0000", 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043328158999999998 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384445+0000", 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.951e-06 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20409 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.42396596, 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043398471000000001, 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20409 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341125+0000", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5799999999999998e-07 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341130+0000", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341145+0000", 2024-09-22T01:14:44.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5386000000000001e-05 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341157+0000", 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1423999999999999e-05 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384515+0000", 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358514000000001 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384523+0000", 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3950000000000003e-06 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20561 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.422445107, 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041926151000000002, 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20561 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342648+0000", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3289999999999998e-06 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342653+0000", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342669+0000", 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6081000000000001e-05 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342681+0000", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1382999999999999e-05 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384562+0000", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041881599999999998 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384571+0000", 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9730000000000003e-06 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.836 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20562 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.422377343000001, 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041931077999999997, 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20562 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342714+0000", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000004e-07 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342719+0000", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2549999999999997e-06 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342736+0000", 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7144999999999999e-05 2024-09-22T01:14:44.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342747+0000", 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1080999999999999e-05 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384631+0000", 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041884059000000001 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384644+0000", 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2619999999999999e-05 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20563 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.422326991, 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041909158000000002, 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:44.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20563 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:44.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342764+0000", 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.068e-06 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342779+0000", 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5302999999999999e-05 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342796+0000", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6798000000000001e-05 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342807+0000", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1024e-05 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384665+0000", 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041858039999999999 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:44.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384672+0000", 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:44.845 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:44.847 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:44.847 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:44.844+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:44.848 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.848 INFO:teuthology.orchestra.run.smithi096.stdout:1971 read 0x429c32f thru 0x42a827e (0xbf50 bytes) 2024-09-22T01:14:44.848 INFO:teuthology.orchestra.run.smithi096.stdout:1972 write 0x8158972 thru 0x815f9df (0x706e bytes) 2024-09-22T01:14:44.848 INFO:teuthology.orchestra.run.smithi096.stdout:1973 read 0x83a23ca thru 0x83a4ec0 (0x2af7 bytes) 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:1975 punch from 0x53d1e34 to 0x53df765, (0xd931 bytes) 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:1976 write 0x3a2545 thru 0x3aa379 (0x7e35 bytes) 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:1977 write 0x6a0b8be thru 0x6a12cb0 (0x73f3 bytes) 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:1978 read 0x45e2ebc thru 0x45ee37e (0xb4c3 bytes) 2024-09-22T01:14:44.849 INFO:teuthology.orchestra.run.smithi096.stdout:1979 trunc from 0xa4c192c to 0x758f441 2024-09-22T01:14:44.850 INFO:teuthology.orchestra.run.smithi096.stdout:1980 trunc from 0x758f441 to 0xe8db6b6 2024-09-22T01:14:44.850 INFO:teuthology.orchestra.run.smithi096.stdout:1982 read 0x5ab57f thru 0x5b2d55 (0x77d7 bytes) 2024-09-22T01:14:44.850 INFO:teuthology.orchestra.run.smithi096.stdout:1983 write 0xb14489f thru 0xb154298 (0xf9fa bytes) 2024-09-22T01:14:44.850 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.850 INFO:teuthology.orchestra.run.smithi096.stdout:1986 trunc from 0xe8db6b6 to 0xab85bce 2024-09-22T01:14:44.857 INFO:teuthology.orchestra.run.smithi096.stdout:1987 trunc from 0xab85bce to 0x54eab7b 2024-09-22T01:14:44.874 INFO:teuthology.orchestra.run.smithi096.stdout:1988 punch from 0x6521c3 to 0x661628, (0xf465 bytes) 2024-09-22T01:14:44.875 INFO:teuthology.orchestra.run.smithi096.stdout:1991 read 0x3f9b040 thru 0x3fa2f8a (0x7f4b bytes) 2024-09-22T01:14:44.875 INFO:teuthology.orchestra.run.smithi096.stdout:1992 write 0x703533 thru 0x709e44 (0x6912 bytes) 2024-09-22T01:14:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:1993 writesame 0xd532b98 thru 0xd537420 data_size 0x1(0x4889 bytes) 2024-09-22T01:14:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:1995 write 0x8430f88 thru 0x8433eef (0x2f68 bytes) 2024-09-22T01:14:44.916 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.916 INFO:teuthology.orchestra.run.smithi096.stdout:1997 write 0x550d62f thru 0x550fb56 (0x2528 bytes) 2024-09-22T01:14:44.921 INFO:teuthology.orchestra.run.smithi096.stdout:1998 read 0x10ad5a7 thru 0x10b8a32 (0xb48c bytes) 2024-09-22T01:14:44.922 INFO:teuthology.orchestra.run.smithi096.stdout:2001 writesame 0xabb344e thru 0xabc1e5b data_size 0x1(0xea0e bytes) 2024-09-22T01:14:44.947 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:44.946+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:44.995 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:44.995 INFO:teuthology.orchestra.run.smithi096.stdout:2004 read 0x28c23c2 thru 0x28d0966 (0xe5a5 bytes) 2024-09-22T01:14:44.996 INFO:teuthology.orchestra.run.smithi096.stdout:2005 punch from 0x538b0d3 to 0x5395504, (0xa431 bytes) 2024-09-22T01:14:45.018 INFO:teuthology.orchestra.run.smithi096.stdout:2006 trunc from 0xd537421 to 0x4b575a0 2024-09-22T01:14:45.019 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:45.019 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:45.019 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:45.019 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:45.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:45.071 INFO:teuthology.orchestra.run.smithi096.stdout:2007 trunc from 0x4b575a0 to 0xce97518 2024-09-22T01:14:45.081 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:45.080+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:45.082 INFO:teuthology.orchestra.run.smithi096.stdout:2008 punch from 0x4433069 to 0x4434202, (0x1199 bytes) 2024-09-22T01:14:45.082 INFO:teuthology.orchestra.run.smithi096.stdout:2009 read 0x9e4cca3 thru 0x9e56654 (0x99b2 bytes) 2024-09-22T01:14:45.084 INFO:teuthology.orchestra.run.smithi096.stdout:2010 trunc from 0xce97518 to 0xe28bf83 2024-09-22T01:14:45.097 INFO:teuthology.orchestra.run.smithi096.stdout:2011 punch from 0xaf90d28 to 0xaf927a4, (0x1a7c bytes) 2024-09-22T01:14:45.097 INFO:teuthology.orchestra.run.smithi096.stdout:2012 read 0x8baaaaf thru 0x8bb58e8 (0xae3a bytes) 2024-09-22T01:14:45.101 INFO:teuthology.orchestra.run.smithi096.stdout:2014 trunc from 0xe28bf83 to 0xc4e544e 2024-09-22T01:14:45.121 INFO:teuthology.orchestra.run.smithi096.stdout:2015 trunc from 0xc4e544e to 0xeb00695 2024-09-22T01:14:45.138 INFO:teuthology.orchestra.run.smithi096.stdout:2016 writesame 0x514e235 thru 0x515c4d3 data_size 0x29(0xe29f bytes) 2024-09-22T01:14:45.147 INFO:teuthology.orchestra.run.smithi096.stdout:2018 write 0x30a6ce7 thru 0x30b266e (0xb988 bytes) 2024-09-22T01:14:45.149 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:45.146+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:45.158 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:45.158 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:45.159 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:45.159 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:45.159 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:45.169 INFO:teuthology.orchestra.run.smithi096.stdout:2020 trunc from 0xeb00695 to 0x92f8034 2024-09-22T01:14:45.173 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:45.182 INFO:teuthology.orchestra.run.smithi096.stdout:2022 read 0x51fa80c thru 0x5207615 (0xce0a bytes) 2024-09-22T01:14:45.182 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:45.182 INFO:teuthology.orchestra.run.smithi096.stdout:2024 read 0x361888e thru 0x3622ef7 (0xa66a bytes) 2024-09-22T01:14:45.183 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:45.208 INFO:teuthology.orchestra.run.smithi096.stdout:2031 write 0xd7cb260 thru 0xd7d4260 (0x9001 bytes) 2024-09-22T01:14:45.215 INFO:teuthology.orchestra.run.smithi096.stdout:2032 clone 19 order 21 su 16384 sc 9 2024-09-22T01:14:45.261 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:45.260+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:45.343 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:45.343 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:45.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:45.343 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027941255999998, 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:14:45.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:14:45.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:14:45.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027889036000001, 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:14:45.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027689387999999, 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:14:45.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027610684999999, 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027550691999998, 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:14:45.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:14:45.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027478199000001, 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:14:45.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:14:45.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027409887000001, 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:14:45.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:14:45.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027351963000001, 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:14:45.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.027296073999999, 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:14:45.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:14:45.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.383 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.140351297, 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:45.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:14:45.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:14:45.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.444984316999999, 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:14:45.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.052859306, 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.3610050579999999, 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:14:45.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:14:45.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:14:45.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.3608868390000008, 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:14:45.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:14:45.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.3583598979999998, 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:14:45.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:45.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.3399135449999999, 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:14:45.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:14:45.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.1802866519999995, 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:14:45.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:14:45.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:14:45.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.1802325570000001, 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:14:45.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.1801462610000009, 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.1795165520000008, 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:45.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:14:45.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:14:45.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:45.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:45.444 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:45.445 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:45.446 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:45.442+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:45.546 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:45.542+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:45.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:45.593 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:14:45.621 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:45.621 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:45.621 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:45.621 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:45.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:45.685 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:45.684+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:45.727 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:45.727 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:45.745 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:45.745 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:45.747 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:45.746+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:45.756 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:45.757 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:45.757 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:45.757 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:45.757 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:45.759 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone18 from 0xd7d4261 (overlap 0xa6c22d) to 0x640e1b 2024-09-22T01:14:45.764 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:45.764 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:45.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:45.783 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:45.783 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:45.802 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:45.802 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:45.820 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:45.820 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:45.840 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:45.841 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:45.859 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:45.860 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:45.860 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:45.860+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:45.940 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:45.940 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:45.949 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:45.946+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:45.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:46.025 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-22T01:14:46.050 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:46.048+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:46.080 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.080 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.096 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:46.151 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:46.148+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:46.160 INFO:teuthology.orchestra.run.smithi096.stdout:2034 read 0xed10d3 thru 0xed6058 (0x4f86 bytes) 2024-09-22T01:14:46.162 INFO:teuthology.orchestra.run.smithi096.stdout:2035 punch from 0x7d97ab6 to 0x7d9b192, (0x36dc bytes) 2024-09-22T01:14:46.167 INFO:teuthology.orchestra.run.smithi096.stdout:2036 write 0x815dcad thru 0x81694b6 (0xb80a bytes) 2024-09-22T01:14:46.177 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:46.177 INFO:teuthology.orchestra.run.smithi096.stdout:2041 punch from 0x2d94c97 to 0x2da3290, (0xe5f9 bytes) 2024-09-22T01:14:46.177 INFO:teuthology.orchestra.run.smithi096.stdout:2043 write 0xa24be6 thru 0xa29807 (0x4c22 bytes) 2024-09-22T01:14:46.215 INFO:teuthology.orchestra.run.smithi096.stdout:2044 writesame 0x5a92ced thru 0x5a98fe5 data_size 0x1(0x62f9 bytes) 2024-09-22T01:14:46.226 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.226 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.243 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:46.252 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:46.248+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:46.254 INFO:teuthology.orchestra.run.smithi096.stdout:2045 trunc from 0xd7d4261 to 0x1e9352f 2024-09-22T01:14:46.296 INFO:teuthology.orchestra.run.smithi096.stdout:2046 writesame 0x3559ea6 thru 0x35675ca data_size 0xb(0xd725 bytes) 2024-09-22T01:14:46.301 INFO:teuthology.orchestra.run.smithi096.stdout:2047 trunc from 0x35675cb to 0x5ea48f 2024-09-22T01:14:46.352 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:46.350+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:46.388 INFO:teuthology.orchestra.run.smithi096.stdout:2048 read 0x5108ca thru 0x51d0aa (0xc7e1 bytes) 2024-09-22T01:14:46.389 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:46.389 INFO:teuthology.orchestra.run.smithi096.stdout:2050 clone 20 order 20 su 32768 sc 15 2024-09-22T01:14:46.392 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.392 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:46.454 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:46.450+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:46.555 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:46.552+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:46.572 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.572 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:46.656 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:46.652+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:46.714 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.714 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:46.775 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:46.774+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:46.851 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone19 intact 2024-09-22T01:14:46.857 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:46.854+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:46.864 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:46.864 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:46.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:46.958 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:46.955+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:47.054 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.054 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.059 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:47.055+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:47.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:47.159 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:47.158+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:47.188 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.188 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.193 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-22T01:14:47.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:47.252 INFO:teuthology.orchestra.run.smithi096.stdout:2051 trunc from 0x5ea48f to 0xdef64ff 2024-09-22T01:14:47.260 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:47.258+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:47.269 INFO:teuthology.orchestra.run.smithi096.stdout:2052 punch from 0x804d207 to 0x8059753, (0xc54c bytes) 2024-09-22T01:14:47.269 INFO:teuthology.orchestra.run.smithi096.stdout:2053 writesame 0x871ae98 thru 0x8721a4c data_size 0x5b(0x6bb5 bytes) 2024-09-22T01:14:47.276 INFO:teuthology.orchestra.run.smithi096.stdout:2054 write 0x2e7e61d thru 0x2e85c83 (0x7667 bytes) 2024-09-22T01:14:47.284 INFO:teuthology.orchestra.run.smithi096.stdout:2056 writesame 0x8da80b8 thru 0x8da89c6 data_size 0x1(0x90f bytes) 2024-09-22T01:14:47.289 INFO:teuthology.orchestra.run.smithi096.stdout:2057 read 0x9cd6531 thru 0x9ce5432 (0xef02 bytes) 2024-09-22T01:14:47.289 INFO:teuthology.orchestra.run.smithi096.stdout:2058 writesame 0x69deff8 thru 0x69ed106 data_size 0x45(0xe10f bytes) 2024-09-22T01:14:47.300 INFO:teuthology.orchestra.run.smithi096.stdout:2059 write 0x5778741 thru 0x577f266 (0x6b26 bytes) 2024-09-22T01:14:47.307 INFO:teuthology.orchestra.run.smithi096.stdout:2060 write 0x15d39e4 thru 0x15d3cb1 (0x2ce bytes) 2024-09-22T01:14:47.326 INFO:teuthology.orchestra.run.smithi096.stdout:2061 trunc from 0xdef64ff to 0x33d2131 2024-09-22T01:14:47.349 INFO:teuthology.orchestra.run.smithi096.stdout:2062 writesame 0x2e1b614 thru 0x2e1c79b data_size 0x2c(0x1188 bytes) 2024-09-22T01:14:47.354 INFO:teuthology.orchestra.run.smithi096.stdout:2063 read 0x25f4037 thru 0x25fe0d9 (0xa0a3 bytes) 2024-09-22T01:14:47.354 INFO:teuthology.orchestra.run.smithi096.stdout:2064 clone 21 order 24 su 131072 sc 5 2024-09-22T01:14:47.360 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:47.359+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:47.383 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.383 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.397 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:47.462 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:47.459+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:47.561 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.561 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.562 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:47.559+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:47.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:47.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:14:47.663 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:47.659+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:47.691 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:47.691 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:47.691 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-22T01:14:47.691 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:14:47.691 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:14:47.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-22T01:14:47.712 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.712 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:47.775 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:47.774+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:47.820 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:47.820 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-09-22T01:14:47.820 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:47.852 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone20 from 0x33d2131 (overlap 0x5ea48f) to 0x1c6491 2024-09-22T01:14:47.865 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:47.863+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:47.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:47.925 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:47.925 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:47.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:47.991 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:47.990+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:48.041 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:48.041 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:48.058 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:48.058 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:48.067 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:48.063+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:48.075 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:48.075 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:48.086 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:48.086 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:48.091 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:48.091 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:48.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:48.111 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:48.112 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:48.130 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:48.150 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:48.150 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:48.171 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:48.172 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:48.184 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-22T01:14:48.189 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:48.187+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:48.219 INFO:teuthology.orchestra.run.smithi096.stdout:2065 write 0x6626a6d thru 0x662ad70 (0x4304 bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2067 punch from 0x34c06b7 to 0x34c96f0, (0x9039 bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2069 read 0x6256cfe thru 0x625a3da (0x36dd bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2070 read 0x2d01742 thru 0x2d0e134 (0xc9f3 bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2072 read 0x2ee95bc thru 0x2eed387 (0x3dcc bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2073 read 0x1130605 thru 0x113451b (0x3f17 bytes) 2024-09-22T01:14:48.223 INFO:teuthology.orchestra.run.smithi096.stdout:2074 writesame 0x26ec766 thru 0x26ee2ed data_size 0x4(0x1b88 bytes) 2024-09-22T01:14:48.247 INFO:teuthology.orchestra.run.smithi096.stdout:2075 trunc from 0x662ad71 to 0x2a7acd9 2024-09-22T01:14:48.269 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:48.267+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:48.277 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:48.278 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:48.278 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:48.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:48.295 INFO:teuthology.orchestra.run.smithi096.stdout:2077 read 0x8893b4 thru 0x88ecad (0x58fa bytes) 2024-09-22T01:14:48.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.295 INFO:teuthology.orchestra.run.smithi096.stdout:2080 read 0x1cdd008 thru 0x1cde8ad (0x18a6 bytes) 2024-09-22T01:14:48.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.308 INFO:teuthology.orchestra.run.smithi096.stdout:2082 trunc from 0x2a7acd9 to 0xab9fbdf 2024-09-22T01:14:48.316 INFO:teuthology.orchestra.run.smithi096.stdout:2083 read 0xa17128d thru 0xa17cb67 (0xb8db bytes) 2024-09-22T01:14:48.316 INFO:teuthology.orchestra.run.smithi096.stdout:2084 read 0x4375903 thru 0x4382095 (0xc793 bytes) 2024-09-22T01:14:48.317 INFO:teuthology.orchestra.run.smithi096.stdout:2085 punch from 0xafd7ca to 0xb0285f, (0x5095 bytes) 2024-09-22T01:14:48.327 INFO:teuthology.orchestra.run.smithi096.stdout:2086 write 0xbaa45f2 thru 0xbaa7908 (0x3317 bytes) 2024-09-22T01:14:48.332 INFO:teuthology.orchestra.run.smithi096.stdout:2087 punch from 0x1ceb17b to 0x1cf250f, (0x7394 bytes) 2024-09-22T01:14:48.332 INFO:teuthology.orchestra.run.smithi096.stdout:2088 write 0x3322e04 thru 0x332fe33 (0xd030 bytes) 2024-09-22T01:14:48.335 INFO:teuthology.orchestra.run.smithi096.stdout:2089 writesame 0x1b9acf6 thru 0x1b9c08a data_size 0x3(0x1395 bytes) 2024-09-22T01:14:48.339 INFO:teuthology.orchestra.run.smithi096.stdout:2090 writesame 0x6dc1ac1 thru 0x6dc855b data_size 0xb(0x6a9b bytes) 2024-09-22T01:14:48.346 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.346 INFO:teuthology.orchestra.run.smithi096.stdout:2095 write 0x270480d thru 0x2706a1c (0x2210 bytes) 2024-09-22T01:14:48.352 INFO:teuthology.orchestra.run.smithi096.stdout:2096 trunc from 0xbaa7909 to 0x9dd0bd3 2024-09-22T01:14:48.369 INFO:teuthology.orchestra.run.smithi096.stdout:2098 read 0x136446 thru 0x13e6be (0x8279 bytes) 2024-09-22T01:14:48.381 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:48.379+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:48.390 INFO:teuthology.orchestra.run.smithi096.stdout:2099 trunc from 0x9dd0bd3 to 0xa5bffe 2024-09-22T01:14:48.411 INFO:teuthology.orchestra.run.smithi096.stdout:2100 punch from 0xefba3 to 0xfdce2, (0xe13f bytes) 2024-09-22T01:14:48.448 INFO:teuthology.orchestra.run.smithi096.stdout:2101 write 0xbce8332 thru 0xbceea47 (0x6716 bytes) 2024-09-22T01:14:48.456 INFO:teuthology.orchestra.run.smithi096.stdout:2102 writesame 0x26d070b thru 0x26dc883 data_size 0x1(0xc179 bytes) 2024-09-22T01:14:48.466 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:48.466 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:48.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:48.466 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:48.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111694356000001, 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:48.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111628190000001, 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:48.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111568153, 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.478 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111533090999998, 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.487 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111502820999998, 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:48.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111473247999999, 2024-09-22T01:14:48.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.496 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111419157, 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:48.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:48.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111363312999998, 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:48.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:48.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:48.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111305423000001, 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:48.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:48.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111243689999998, 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:48.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.515 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111148227000001, 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:48.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:48.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:48.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111086404000002, 2024-09-22T01:14:48.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:48.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:48.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110995264, 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110933358, 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.530 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:48.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:48.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110872871000002, 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:48.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:48.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110815401, 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:48.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110758121, 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:48.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110698596999999, 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:48.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:48.553 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.110645713, 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:48.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:48.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.101533151000002, 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:48.558 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:48.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:48.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:48.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:48.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:48.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:48.564 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:48.565 INFO:teuthology.orchestra.run.smithi096.stdout:2104 read 0x19c4fda thru 0x19d43d7 (0xf3fe bytes) 2024-09-22T01:14:48.565 INFO:teuthology.orchestra.run.smithi096.stdout:2106 write 0x2a1950 thru 0x2adfa1 (0xc652 bytes) 2024-09-22T01:14:48.565 INFO:teuthology.orchestra.run.smithi096.stdout:2107 write 0x5067f20 thru 0x506bdac (0x3e8d bytes) 2024-09-22T01:14:48.565 INFO:teuthology.orchestra.run.smithi096.stdout:2108 write 0x7b1f382 thru 0x7b265a9 (0x7228 bytes) 2024-09-22T01:14:48.566 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:48.567 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:48.563+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:48.567 INFO:teuthology.orchestra.run.smithi096.stdout:2109 read 0x399938c thru 0x399d9b4 (0x4629 bytes) 2024-09-22T01:14:48.582 INFO:teuthology.orchestra.run.smithi096.stdout:2110 trunc from 0xbceea48 to 0x6a826b6 2024-09-22T01:14:48.616 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:14:48.623 INFO:teuthology.orchestra.run.smithi096.stdout:2111 writesame 0x65cf544 thru 0x65d7737 data_size 0x2(0x81f4 bytes) 2024-09-22T01:14:48.641 INFO:teuthology.orchestra.run.smithi096.stdout:2112 write 0x3f89861 thru 0x3f8c1cf (0x296f bytes) 2024-09-22T01:14:48.645 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.646 INFO:teuthology.orchestra.run.smithi096.stdout:2116 writesame 0x43ef9fe thru 0x43f246d data_size 0x1c(0x2a70 bytes) 2024-09-22T01:14:48.648 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.648 INFO:teuthology.orchestra.run.smithi096.stdout:2118 punch from 0x65f7ee9 to 0x65fa026, (0x213d bytes) 2024-09-22T01:14:48.657 INFO:teuthology.orchestra.run.smithi096.stdout:2119 trunc from 0x6a826b6 to 0x208fceb 2024-09-22T01:14:48.665 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:48.662+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:48.722 INFO:teuthology.orchestra.run.smithi096.stdout:2121 writesame 0xd1de9c1 thru 0xd1ec2a6 data_size 0x1(0xd8e6 bytes) 2024-09-22T01:14:48.722 INFO:teuthology.orchestra.run.smithi096.stdout:2122 writesame 0x709047f thru 0x709e430 data_size 0x89(0xdfb2 bytes) 2024-09-22T01:14:48.730 INFO:teuthology.orchestra.run.smithi096.stdout:2124 write 0xadf971b thru 0xae0388d (0xa173 bytes) 2024-09-22T01:14:48.739 INFO:teuthology.orchestra.run.smithi096.stdout:2125 write 0x55c70b thru 0x56868b (0xbf81 bytes) 2024-09-22T01:14:48.748 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:48.748 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:48.748 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:48.748 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:48.758 INFO:teuthology.orchestra.run.smithi096.stdout:2126 trunc from 0xd1ec2a7 to 0x6481b4c 2024-09-22T01:14:48.762 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:48.780 INFO:teuthology.orchestra.run.smithi096.stdout:2127 punch from 0x430c348 to 0x430f7eb, (0x34a3 bytes) 2024-09-22T01:14:48.807 INFO:teuthology.orchestra.run.smithi096.stdout:2129 write 0xd5738d9 thru 0xd57b18e (0x78b6 bytes) 2024-09-22T01:14:48.809 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:48.807+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:48.813 INFO:teuthology.orchestra.run.smithi096.stdout:2131 read 0x5ccc8d2 thru 0x5ccf06c (0x279b bytes) 2024-09-22T01:14:48.820 INFO:teuthology.orchestra.run.smithi096.stdout:2132 trunc from 0xd57b18f to 0x9cb29af 2024-09-22T01:14:48.855 INFO:teuthology.orchestra.run.smithi096.stdout:2133 writesame 0xa027210 thru 0xa028ef5 data_size 0x1b(0x1ce6 bytes) 2024-09-22T01:14:48.855 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.855 INFO:teuthology.orchestra.run.smithi096.stdout:2136 punch from 0x272ecf to 0x27818f, (0x52c0 bytes) 2024-09-22T01:14:48.867 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:48.862+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:48.872 INFO:teuthology.orchestra.run.smithi096.stdout:2137 writesame 0xbe48d23 thru 0xbe4f7e8 data_size 0x2(0x6ac6 bytes) 2024-09-22T01:14:48.872 INFO:teuthology.orchestra.run.smithi096.stdout:2138 read 0x2f94c63 thru 0x2fa3ddf (0xf17d bytes) 2024-09-22T01:14:48.873 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.873 INFO:teuthology.orchestra.run.smithi096.stdout:2140 read 0x3445202 thru 0x34490d5 (0x3ed4 bytes) 2024-09-22T01:14:48.874 INFO:teuthology.orchestra.run.smithi096.stdout:2141 read 0x90fd97f thru 0x9101c1c (0x429e bytes) 2024-09-22T01:14:48.875 INFO:teuthology.orchestra.run.smithi096.stdout:2142 read 0x723c9e8 thru 0x72444dd (0x7af6 bytes) 2024-09-22T01:14:48.875 INFO:teuthology.orchestra.run.smithi096.stdout:2143 read 0x84f6672 thru 0x84fb999 (0x5328 bytes) 2024-09-22T01:14:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:48.886 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:48.892 INFO:teuthology.orchestra.run.smithi096.stdout:2144 write 0xca95a80 thru 0xca9b67d (0x5bfe bytes) 2024-09-22T01:14:48.899 INFO:teuthology.orchestra.run.smithi096.stdout:2145 punch from 0x4e5a576 to 0x4e5acb4, (0x73e bytes) 2024-09-22T01:14:48.900 INFO:teuthology.orchestra.run.smithi096.stdout:2146 punch from 0x2da1e4b to 0x2dab277, (0x942c bytes) 2024-09-22T01:14:48.900 INFO:teuthology.orchestra.run.smithi096.stdout:2150 punch from 0x74357dc to 0x743accd, (0x54f1 bytes) 2024-09-22T01:14:48.901 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:48.915 INFO:teuthology.orchestra.run.smithi096.stdout:2152 trunc from 0xca9b67e to 0x73e1abd 2024-09-22T01:14:48.943 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:48.943 INFO:teuthology.orchestra.run.smithi096.stdout:2154 write 0x2944ca6 thru 0x2945a1d (0xd78 bytes) 2024-09-22T01:14:48.967 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:48.966+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:48.974 INFO:teuthology.orchestra.run.smithi096.stdout:2155 write 0xcfc698b thru 0xcfce685 (0x7cfb bytes) 2024-09-22T01:14:48.980 INFO:teuthology.orchestra.run.smithi096.stdout:2156 write 0x787ed8f thru 0x7880839 (0x1aab bytes) 2024-09-22T01:14:48.987 INFO:teuthology.orchestra.run.smithi096.stdout:2157 punch from 0x91b0c59 to 0x91b5fd5, (0x537c bytes) 2024-09-22T01:14:48.987 INFO:teuthology.orchestra.run.smithi096.stdout:2158 read 0x6dde674 thru 0x6deb267 (0xcbf4 bytes) 2024-09-22T01:14:48.988 INFO:teuthology.orchestra.run.smithi096.stdout:2159 write 0xf2ce54 thru 0xf33bd6 (0x6d83 bytes) 2024-09-22T01:14:48.991 INFO:teuthology.orchestra.run.smithi096.stdout:2160 read 0x1d93125 thru 0x1d9bf72 (0x8e4e bytes) 2024-09-22T01:14:48.992 INFO:teuthology.orchestra.run.smithi096.stdout:2161 write 0x2454bfc thru 0x245fbb9 (0xafbe bytes) 2024-09-22T01:14:48.995 INFO:teuthology.orchestra.run.smithi096.stdout:2162 read 0x54bd106 thru 0x54bd1e6 (0xe1 bytes) 2024-09-22T01:14:48.996 INFO:teuthology.orchestra.run.smithi096.stdout:2163 punch from 0x44b730f to 0x44bca19, (0x570a bytes) 2024-09-22T01:14:48.996 INFO:teuthology.orchestra.run.smithi096.stdout:2165 clone 22 order 18 su 1024 sc 8 2024-09-22T01:14:49.068 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:49.063+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20393 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 2883584~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.735088999999999, 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042022144999999997, 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20393 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340294+0000", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340295+0000", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5799999999999998e-07 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340299+0000", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1540000000000004e-06 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340313+0000", 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3854e-05 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340327+0000", 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4151000000000001e-05 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381789+0000", 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041462496000000001 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.381873+0000", 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4049999999999997e-05 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382316+0000", 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044248200000000001 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20394 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3211264~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.735040900000001, 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042595531999999998, 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20394 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340342+0000", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340343+0000", 2024-09-22T01:14:49.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.95e-07 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340348+0000", 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0139999999999998e-06 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340362+0000", 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4744e-05 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340373+0000", 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0411000000000001e-05 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382394+0000", 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042020841000000003 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382511+0000", 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000117518 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382937+0000", 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042600899999999998 2024-09-22T01:14:49.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20395 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3538944~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734995413, 2024-09-22T01:14:49.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043113203000000003, 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20395 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340387+0000", 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340388+0000", 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8100000000000001e-07 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340393+0000", 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7679999999999999e-06 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340408+0000", 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4924e-05 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340420+0000", 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2476e-05 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.382978+0000", 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042557375000000001 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383074+0000", 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5839000000000002e-05 2024-09-22T01:14:49.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383500+0000", 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00042684 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20396 3.1e 3:790389b4:::rbd_data.11249227f48c.0000000000000005:e [read 3899392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734947825999999, 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043547334, 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20396 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340435+0000", 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340436+0000", 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4300000000000001e-07 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340456+0000", 2024-09-22T01:14:49.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0012000000000001e-05 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340472+0000", 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6302000000000001e-05 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340482+0000", 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0453e-05 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383550+0000", 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043067831000000001 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383628+0000", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8008000000000003e-05 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.383982+0000", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00035378500000000002 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20397 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:49.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734883764999999, 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043522706000000001, 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20397 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340499+0000", 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340500+0000", 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3200000000000003e-07 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340505+0000", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2100000000000001e-06 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340520+0000", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4865000000000001e-05 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340544+0000", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3708000000000001e-05 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384013+0000", 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043469926999999998 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384022+0000", 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0639999999999994e-06 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20398 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.105 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734822952000002, 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043503679000000003, 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20398 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340560+0000", 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340561+0000", 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340566+0000", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1619999999999999e-06 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340580+0000", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4633e-05 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340592+0000", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1134e-05 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384056+0000", 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043464964000000002 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384063+0000", 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20399 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734773140000001, 2024-09-22T01:14:49.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043483817000000001, 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20399 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340609+0000", 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340611+0000", 2024-09-22T01:14:49.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.28e-06 2024-09-22T01:14:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340616+0000", 2024-09-22T01:14:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0849999999999996e-06 2024-09-22T01:14:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340631+0000", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5384999999999999e-05 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340642+0000", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0644e-05 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384087+0000", 2024-09-22T01:14:49.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043445399000000003 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384093+0000", 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20400 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734726416000001, 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043466537, 2024-09-22T01:14:49.116 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20400 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340656+0000", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340657+0000", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.85e-07 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340662+0000", 2024-09-22T01:14:49.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1279999999999999e-06 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340677+0000", 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5279999999999999e-05 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340688+0000", 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0689999999999999e-05 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384117+0000", 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043428384 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384123+0000", 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1700000000000002e-06 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20401 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734679710000002, 2024-09-22T01:14:49.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043449197000000002, 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20401 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340703+0000", 2024-09-22T01:14:49.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340704+0000", 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0890000000000001e-06 2024-09-22T01:14:49.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340718+0000", 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3804e-05 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340733+0000", 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5013e-05 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340744+0000", 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1207e-05 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384145+0000", 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043401020999999998 2024-09-22T01:14:49.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384152+0000", 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.063e-06 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20402 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734622818999998, 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043424268000000002, 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20402 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340760+0000", 2024-09-22T01:14:49.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340761+0000", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1699999999999997e-07 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340766+0000", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.215e-06 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340781+0000", 2024-09-22T01:14:49.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5065000000000001e-05 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340792+0000", 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0556e-05 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384177+0000", 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043385328000000001 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384184+0000", 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.187e-06 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20403 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734576358999998, 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043422686000000002, 2024-09-22T01:14:49.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20403 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340806+0000", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340807+0000", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.47e-07 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340812+0000", 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:14:49.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340828+0000", 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5553000000000002e-05 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340837+0000", 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3870000000000001e-06 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384220+0000", 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043383163000000002 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384229+0000", 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6489999999999997e-06 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20404 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734529752, 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043441768999999998, 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20404 2024-09-22T01:14:49.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340853+0000", 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340854+0000", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7899999999999997e-07 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340859+0000", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9969999999999998e-06 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340874+0000", 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5404e-05 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340885+0000", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0916e-05 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384283+0000", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043397463999999997 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384295+0000", 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2109e-05 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20405 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734481166999998, 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043420168000000002, 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20405 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340901+0000", 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340903+0000", 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.372e-06 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340908+0000", 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1560000000000003e-06 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340924+0000", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5829999999999999e-05 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340935+0000", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0951000000000001e-05 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384316+0000", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043380836999999998 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384322+0000", 2024-09-22T01:14:49.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0220000000000001e-06 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20406 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734432395999999, 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043418238999999997, 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20406 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340950+0000", 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340952+0000", 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3030000000000001e-06 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340956+0000", 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7790000000000003e-06 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340971+0000", 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5196e-05 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.340997+0000", 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5863999999999998e-05 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384356+0000", 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358529999999999 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384368+0000", 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2567e-05 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20407 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734369432000001, 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043399115000000002, 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20407 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341013+0000", 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341014+0000", 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0160000000000001e-06 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341031+0000", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6769000000000002e-05 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341047+0000", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341058+0000", 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1079e-05 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384406+0000", 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043347572000000001 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384412+0000", 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2439999999999998e-06 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20408 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.152 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734306966999998, 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043369389000000001, 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20408 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341076+0000", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341077+0000", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2020000000000001e-06 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341082+0000", 2024-09-22T01:14:49.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8879999999999996e-06 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341099+0000", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7269999999999999e-05 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341110+0000", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0919e-05 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384438+0000", 2024-09-22T01:14:49.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043328158999999998 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384445+0000", 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.951e-06 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20409 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.734258454999999, 2024-09-22T01:14:49.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043398471000000001, 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20409 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:49.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341124+0000", 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341125+0000", 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5799999999999998e-07 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341130+0000", 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341145+0000", 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5386000000000001e-05 2024-09-22T01:14:49.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.341157+0000", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1423999999999999e-05 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384515+0000", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.043358514000000001 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384523+0000", 2024-09-22T01:14:49.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3950000000000003e-06 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20561 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.732737602, 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041926151000000002, 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20561 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342645+0000", 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342648+0000", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3289999999999998e-06 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342653+0000", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7849999999999999e-06 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342669+0000", 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6081000000000001e-05 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342681+0000", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1382999999999999e-05 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384562+0000", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041881599999999998 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384571+0000", 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9730000000000003e-06 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20562 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.732669838, 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041931077999999997, 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20562 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.165 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342713+0000", 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342714+0000", 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000004e-07 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342719+0000", 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2549999999999997e-06 2024-09-22T01:14:49.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342736+0000", 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7144999999999999e-05 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342747+0000", 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1080999999999999e-05 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384631+0000", 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041884059000000001 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384644+0000", 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2619999999999999e-05 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20563 3.1e 3.2d91c09e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.732619486000001, 2024-09-22T01:14:49.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041909158000000002, 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20563 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342763+0000", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342764+0000", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.068e-06 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342779+0000", 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5302999999999999e-05 2024-09-22T01:14:49.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342796+0000", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6798000000000001e-05 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.342807+0000", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1024e-05 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384665+0000", 2024-09-22T01:14:49.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.041858039999999999 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.384672+0000", 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9249999999999998e-06 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.173 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:49.175 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:49.175 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:49.175+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:49.276 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:49.274+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:49.344 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:49.344 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:49.345 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:49.345 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:49.359 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:49.377 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:49.374+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:49.477 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:49.474+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:49.482 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:49.483 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:49.483 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:49.483 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:49.483 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:49.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:49.578 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:49.574+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:49.622 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:49.623 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.308311007999997, 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:14:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:14:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:14:49.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:49.630 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.308258788000003, 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:49.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:14:49.634 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.308059139999997, 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.635 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:14:49.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:14:49.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:49.639 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307980436999998, 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:49.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:14:49.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307920443999997, 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.644 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:14:49.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:14:49.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307847950999999, 2024-09-22T01:14:49.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.652 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307779639000003, 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:49.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:14:49.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.657 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307721715, 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.658 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:14:49.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.307665825999997, 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.420721049000001, 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:49.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.725354069000002, 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:14:49.673 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:14:49.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:14:49.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:14:49.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.333229058000001, 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:14:49.679 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:49.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.64137481, 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:49.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:14:49.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.641256590999999, 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:14:49.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.63872965, 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:14:49.696 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:14:49.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:14:49.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:49.702 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.620283297, 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:14:49.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.460656404, 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:14:49.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.460602309, 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:14:49.714 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:14:49.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:14:49.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:14:49.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:14:49.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:49.726 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.460516012999999, 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:14:49.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:14:49.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:14:49.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:14:49.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 13.459886303999999, 2024-09-22T01:14:49.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:49.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:49.739 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:49.740 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:49.741 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:49.738+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:49.842 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:49.838+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:49.877 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:49.878 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:49.878 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:49.878 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:49.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:49.942 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:49.939+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:49.964 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone21 from 0xcfce686 (overlap 0xa5bffe) to 0x637332 2024-09-22T01:14:50.018 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:50.018 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:50.018 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:50.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:50.077 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:50.075+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:50.145 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:50.143+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:50.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:50.205 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:50.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.975752808999999, 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:14:50.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:14:50.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.97567798, 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:14:50.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:50.217 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.975639955999998, 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:14:50.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:50.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:50.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.975590734000001, 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:14:50.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.975554386999999, 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:14:50.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:14:50.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.316681497000001, 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:14:50.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:14:50.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.314500751000001, 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:14:50.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:50.243 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.314428266, 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:50.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.247 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.314366449000001, 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.248 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:14:50.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.91119887, 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.252 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:14:50.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:14:50.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:14:50.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234517089000001, 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:14:50.256 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:14:50.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234461464000001, 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.261 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234358844000001, 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:14:50.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:14:50.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234258356, 2024-09-22T01:14:50.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:14:50.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.273 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234197789, 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:14:50.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:14:50.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:50.278 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234140414000001, 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:50.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.282 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234071704, 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:14:50.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:14:50.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.234009759999999, 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:50.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:50.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.233945793, 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:50.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.3778506530000003, 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:50.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:50.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:50.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:50.300 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:50.301 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:50.299+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:50.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:50.363 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:50.363 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:50.388 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:50.388 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:50.397 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-22T01:14:50.402 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:50.398+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:50.408 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:50.408 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:50.410 INFO:teuthology.orchestra.run.smithi096.stdout:2166 punch from 0x766a6ab to 0x766c21a, (0x1b6f bytes) 2024-09-22T01:14:50.418 INFO:teuthology.orchestra.run.smithi096.stdout:2169 write 0xa3ce776 thru 0xa3d7fde (0x9869 bytes) 2024-09-22T01:14:50.431 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:50.432 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:50.434 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:50.434 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:50.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:50.455 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:50.477 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:50.503 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:50.503 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:50.503 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:50.498+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:50.526 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:50.526 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:50.576 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:50.576 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:50.585 INFO:teuthology.orchestra.run.smithi096.stdout:2171 writesame 0xebeea28 thru 0xebf5365 data_size 0x13(0x693e bytes) 2024-09-22T01:14:50.585 INFO:teuthology.orchestra.run.smithi096.stdout:2172 write 0x4000b62 thru 0x400a172 (0x9611 bytes) 2024-09-22T01:14:50.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:50.640 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:50.638+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:50.641 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:14:50.724 INFO:teuthology.orchestra.run.smithi096.stdout:2173 writesame 0xaaf3f9b thru 0xab02d27 data_size 0x1d(0xed8d bytes) 2024-09-22T01:14:50.724 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:50.722+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:50.783 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:50.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:50.840 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:50.838+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:50.896 INFO:teuthology.orchestra.run.smithi096.stdout:2174 writesame 0xb89abc5 thru 0xb8a66ff data_size 0xd(0xbb3b bytes) 2024-09-22T01:14:50.907 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:50.902+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:50.957 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:50.957 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:50.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:51.008 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:51.003+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:51.108 INFO:teuthology.orchestra.run.smithi096.stdout:2175 write 0x9c09184 thru 0x9c16d6a (0xdbe7 bytes) 2024-09-22T01:14:51.109 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:51.106+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:51.130 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:51.131 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:51.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:51.231 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:51.230+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:51.242 INFO:teuthology.orchestra.run.smithi096.stdout:2176 trunc from 0xebf5366 to 0xe4cf8c5 2024-09-22T01:14:51.252 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:51.252 INFO:teuthology.orchestra.run.smithi096.stdout:2179 write 0x4862b70 thru 0x486c736 (0x9bc7 bytes) 2024-09-22T01:14:51.310 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:51.307+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:51.353 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:51.353 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:51.374 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:51.399 INFO:teuthology.orchestra.run.smithi096.stdout:2180 writesame 0x93a441a thru 0x93afb68 data_size 0x1(0xb74f bytes) 2024-09-22T01:14:51.411 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:51.407+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:51.512 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:51.510+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:51.513 INFO:teuthology.orchestra.run.smithi096.stdout:2181 write 0xcc42a00 thru 0xcc4df17 (0xb518 bytes) 2024-09-22T01:14:51.522 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:51.522 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:51.536 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:51.613 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:51.611+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:51.621 INFO:teuthology.orchestra.run.smithi096.stdout:2183 trunc from 0xe4cf8c5 to 0x9506636 2024-09-22T01:14:51.625 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:14:51.711 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:51.711 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:51.713 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:51.711+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:51.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:51.779 INFO:teuthology.orchestra.run.smithi096.stdout:2184 trunc from 0x9506636 to 0x858bbf5 2024-09-22T01:14:51.814 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:51.810+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:51.842 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:51.842 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:51.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:51.906 INFO:teuthology.orchestra.run.smithi096.stdout:2186 write 0xbe0835d thru 0xbe13a6a (0xb70e bytes) 2024-09-22T01:14:51.915 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:51.914+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:51.921 INFO:teuthology.orchestra.run.smithi096.stdout:2187 read 0xc709 thru 0x11d40 (0x5638 bytes) 2024-09-22T01:14:51.935 INFO:teuthology.orchestra.run.smithi096.stdout:2188 write 0xdf1d451 thru 0xdf1ef2e (0x1ade bytes) 2024-09-22T01:14:51.946 INFO:teuthology.orchestra.run.smithi096.stdout:2189 read 0x336ee9b thru 0x337c225 (0xd38b bytes) 2024-09-22T01:14:51.955 INFO:teuthology.orchestra.run.smithi096.stdout:2190 punch from 0x5f7bc40 to 0x5f7fea8, (0x4268 bytes) 2024-09-22T01:14:51.955 INFO:teuthology.orchestra.run.smithi096.stdout:2191 write 0x5c9d1b3 thru 0x5caa482 (0xd2d0 bytes) 2024-09-22T01:14:52.016 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:52.014+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:52.027 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:52.027 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:52.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:52.053 INFO:teuthology.orchestra.run.smithi096.stdout:2194 writesame 0xbe01f0f thru 0xbe1148e data_size 0x20(0xf580 bytes) 2024-09-22T01:14:52.079 INFO:teuthology.orchestra.run.smithi096.stdout:2195 trunc from 0xdf1ef2f to 0x1faf556 2024-09-22T01:14:52.119 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:52.118+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:52.214 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:52.214 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:52.219 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:52.215+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:52.232 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:52.275 INFO:teuthology.orchestra.run.smithi096.stdout:2196 write 0x21a502c thru 0x21aff66 (0xaf3b bytes) 2024-09-22T01:14:52.292 INFO:teuthology.orchestra.run.smithi096.stdout:2199 writesame 0x1f9bbc1 thru 0x1fab01b data_size 0x1(0xf45b bytes) 2024-09-22T01:14:52.319 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:52.315+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:52.349 INFO:teuthology.orchestra.run.smithi096.stdout:2200 read 0xbb9dd2 thru 0xbbd196 (0x33c5 bytes) 2024-09-22T01:14:52.351 INFO:teuthology.orchestra.run.smithi096.stdout:2201 read 0x1162540 thru 0x1163e00 (0x18c1 bytes) 2024-09-22T01:14:52.353 INFO:teuthology.orchestra.run.smithi096.stdout:2202 read 0x219a4df thru 0x21a5b2d (0xb64f bytes) 2024-09-22T01:14:52.356 INFO:teuthology.orchestra.run.smithi096.stdout:2203 trunc from 0x21aff67 to 0x378ce85 2024-09-22T01:14:52.372 INFO:teuthology.orchestra.run.smithi096.stdout:2204 trunc from 0x378ce85 to 0x90968ae 2024-09-22T01:14:52.382 INFO:teuthology.orchestra.run.smithi096.stdout:2205 punch from 0x56252dc to 0x5627afc, (0x2820 bytes) 2024-09-22T01:14:52.395 INFO:teuthology.orchestra.run.smithi096.stdout:2206 writesame 0xc358111 thru 0xc3661cf data_size 0x25(0xe0bf bytes) 2024-09-22T01:14:52.396 INFO:teuthology.orchestra.run.smithi096.stdout:2207 write 0x5e1e214 thru 0x5e1f8da (0x16c7 bytes) 2024-09-22T01:14:52.403 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:52.403 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:52.411 INFO:teuthology.orchestra.run.smithi096.stdout:2208 write 0xbd83149 thru 0xbd91a6a (0xe922 bytes) 2024-09-22T01:14:52.420 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:52.415+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:52.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:52.427 INFO:teuthology.orchestra.run.smithi096.stdout:2209 punch from 0x6a2bdde to 0x6a3bc21, (0xfe43 bytes) 2024-09-22T01:14:52.427 INFO:teuthology.orchestra.run.smithi096.stdout:2211 writesame 0x3087dbe thru 0x3097da9 data_size 0x16(0xffec bytes) 2024-09-22T01:14:52.442 INFO:teuthology.orchestra.run.smithi096.stdout:2212 read 0xac0317f thru 0xac09444 (0x62c6 bytes) 2024-09-22T01:14:52.442 INFO:teuthology.orchestra.run.smithi096.stdout:2213 write 0x78229c5 thru 0x7828068 (0x56a4 bytes) 2024-09-22T01:14:52.458 INFO:teuthology.orchestra.run.smithi096.stdout:2214 read 0x6509d7e thru 0x65156fe (0xb981 bytes) 2024-09-22T01:14:52.458 INFO:teuthology.orchestra.run.smithi096.stdout:2215 read 0x5fdced6 thru 0x5fe174d (0x4878 bytes) 2024-09-22T01:14:52.459 INFO:teuthology.orchestra.run.smithi096.stdout:2217 read 0x47dcbbe thru 0x47e6775 (0x9bb8 bytes) 2024-09-22T01:14:52.459 INFO:teuthology.orchestra.run.smithi096.stdout:2219 read 0xab6557a thru 0xab7169e (0xc125 bytes) 2024-09-22T01:14:52.469 INFO:teuthology.orchestra.run.smithi096.stdout:2220 write 0xdcf969b thru 0xdd04e49 (0xb7af bytes) 2024-09-22T01:14:52.483 INFO:teuthology.orchestra.run.smithi096.stdout:2221 clone 23 order 19 su 65536 sc 14 2024-09-22T01:14:52.520 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:52.515+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:52.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:52.555 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:52.555 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:52.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:52.621 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:52.618+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:52.690 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:52.691 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:52.707 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:52.707 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:52.722 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:52.718+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:52.723 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:52.724 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:52.738 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:52.738 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:52.741 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:52.741 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:52.753 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:52.759 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:52.759 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:52.776 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:52.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:52.794 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:52.813 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:52.814 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:52.817 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone22 from 0xdd04e4a (overlap 0x1faf556) to 0xcac888 2024-09-22T01:14:52.823 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:52.819+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:52.838 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:52.838 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:52.838 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-22T01:14:52.838 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:14:52.838 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:14:52.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-22T01:14:52.892 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:52.892 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:52.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:52.892 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538306227, 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:52.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538240061, 2024-09-22T01:14:52.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:52.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.901 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538180023999999, 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.902 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538144962000001, 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:52.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.910 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538114692000001, 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.911 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:52.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.915 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538085119000002, 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.920 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.538031027999999, 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:52.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537975184, 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:52.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:52.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537917294, 2024-09-22T01:14:52.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.935 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:52.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:52.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:52.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:52.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537855561000001, 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:52.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:52.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:52.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537760098, 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:52.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537698275, 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:52.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537607135000002, 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:52.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537545228999999, 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.968 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537484742, 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:52.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537427271999999, 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:52.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:52.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537369991999999, 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537310468000001, 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:52.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.537257583999999, 2024-09-22T01:14:52.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:52.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.528145022, 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:52.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:52.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:52.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:53.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.002 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:53.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:53.004 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:52.998+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:53.063 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:14:53.063 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:53.127 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:53.126+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:53.186 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:53.186 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:53.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57672 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175454+0000", 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078369110000000002, 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078538110000000005, 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57672 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.187 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175454+0000", 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175454+0000", 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175455+0000", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4400000000000001e-07 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175457+0000", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.091e-06 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175462+0000", 2024-09-22T01:14:53.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4879999999999998e-06 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175466+0000", 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57704 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176026+0000", 2024-09-22T01:14:53.190 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0072646409999999996, 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007319674, 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57704 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176026+0000", 2024-09-22T01:14:53.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176026+0000", 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176027+0000", 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1600000000000002e-07 2024-09-22T01:14:53.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176028+0000", 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176033+0000", 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5809999999999998e-06 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176037+0000", 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.817e-06 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57170 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183074+0000", 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.000217161, 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302071, 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57170 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183074+0000", 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183074+0000", 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183074+0000", 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183076+0000", 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8640000000000001e-06 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183082+0000", 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3249999999999998e-06 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183086+0000", 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0720000000000001e-06 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57673 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175472+0000", 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0078195220000000006, 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007933275, 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57673 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175472+0000", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175472+0000", 2024-09-22T01:14:53.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175472+0000", 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175474+0000", 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.671e-06 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175478+0000", 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7310000000000001e-06 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175495+0000", 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6779e-05 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57705 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176042+0000", 2024-09-22T01:14:53.201 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00724948, 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.007391239, 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57705 2024-09-22T01:14:53.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176042+0000", 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176042+0000", 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176042+0000", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176044+0000", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7269999999999999e-06 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176048+0000", 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5480000000000003e-06 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176052+0000", 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7850000000000002e-06 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57171 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183121+0000", 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00016986200000000001, 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034344699999999998, 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.205 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57171 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183121+0000", 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183121+0000", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183122+0000", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2200000000000004e-07 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183124+0000", 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8330000000000001e-06 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183129+0000", 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4099999999999999e-06 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183133+0000", 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.014e-06 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57674 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175502+0000", 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007789397, 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079908759999999992, 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57674 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175502+0000", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175502+0000", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175503+0000", 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5269999999999999e-06 2024-09-22T01:14:53.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175505+0000", 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1789999999999998e-06 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175512+0000", 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8950000000000001e-06 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175516+0000", 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1710000000000004e-06 2024-09-22T01:14:53.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57706 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176057+0000", 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0072342379999999996, 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0074635760000000004, 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57706 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176057+0000", 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176057+0000", 2024-09-22T01:14:53.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176057+0000", 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.27e-07 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176062+0000", 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0089999999999998e-06 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176067+0000", 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.079e-06 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176071+0000", 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8010000000000001e-06 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57172 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183162+0000", 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00012950900000000001, 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00038731200000000001, 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57172 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.217 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183162+0000", 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183162+0000", 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183162+0000", 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1399999999999997e-07 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183164+0000", 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9e-06 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183169+0000", 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3360000000000002e-06 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183173+0000", 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0180000000000003e-06 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57675 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175528+0000", 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077627069999999998, 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080520039999999998, 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57675 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175528+0000", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175528+0000", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175529+0000", 2024-09-22T01:14:53.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8999999999999997e-07 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175531+0000", 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0269999999999998e-06 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175536+0000", 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2480000000000004e-06 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175540+0000", 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0060000000000003e-06 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57707 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176076+0000", 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0072148239999999999, 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075311329999999998, 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57707 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176076+0000", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176076+0000", 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176077+0000", 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2399999999999999e-07 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176078+0000", 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.649e-06 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176083+0000", 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7870000000000001e-06 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176087+0000", 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7950000000000001e-06 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57173 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183201+0000", 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 8.9965999999999999e-05, 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043405599999999999, 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57173 2024-09-22T01:14:53.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183201+0000", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183201+0000", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183202+0000", 2024-09-22T01:14:53.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1799999999999995e-07 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183203+0000", 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.809e-06 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183209+0000", 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3920000000000002e-06 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183213+0000", 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.861e-06 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57676 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175545+0000", 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077457189999999999, 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081169399999999996, 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.231 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57676 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175545+0000", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175545+0000", 2024-09-22T01:14:53.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175546+0000", 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175548+0000", 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7600000000000001e-06 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175552+0000", 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7509999999999999e-06 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175556+0000", 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.76e-06 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57708 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176093+0000", 2024-09-22T01:14:53.234 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071983510000000004, 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075967099999999996, 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57708 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176093+0000", 2024-09-22T01:14:53.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176093+0000", 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176093+0000", 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4900000000000001e-07 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176095+0000", 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7430000000000001e-06 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176100+0000", 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7509999999999999e-06 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176103+0000", 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7629999999999998e-06 2024-09-22T01:14:53.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57174 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183245+0000", 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.6037e-05, 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00047165100000000001, 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57174 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183245+0000", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183245+0000", 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183246+0000", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183247+0000", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8870000000000001e-06 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183254+0000", 2024-09-22T01:14:53.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.178e-06 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183258+0000", 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9339999999999999e-06 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57677 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175561+0000", 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077301820000000004, 2024-09-22T01:14:53.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00818318, 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57677 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175561+0000", 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175561+0000", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175561+0000", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1300000000000001e-07 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175563+0000", 2024-09-22T01:14:53.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.606e-06 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175568+0000", 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175571+0000", 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7950000000000001e-06 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57709 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176108+0000", 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071826900000000003, 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0076630070000000003, 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57709 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176108+0000", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176108+0000", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176109+0000", 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1399999999999998e-07 2024-09-22T01:14:53.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176110+0000", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5999999999999999e-06 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176115+0000", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7339999999999999e-06 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176119+0000", 2024-09-22T01:14:53.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6729999999999998e-06 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57175 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183295+0000", 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9999962, 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051053199999999996, 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.248 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57175 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183295+0000", 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183295+0000", 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183296+0000", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183297+0000", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.747e-06 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183303+0000", 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4009999999999996e-06 2024-09-22T01:14:53.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183307+0000", 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57678 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175576+0000", 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0077148329999999999, 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082569089999999998, 2024-09-22T01:14:53.251 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57678 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175576+0000", 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175576+0000", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175577+0000", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1300000000000001e-07 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175578+0000", 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.762e-06 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175583+0000", 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6550000000000003e-06 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175588+0000", 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3600000000000004e-06 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.254 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57710 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176123+0000", 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071676309999999998, 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0077369750000000001, 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57710 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.255 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176123+0000", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176123+0000", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176124+0000", 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.22e-07 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176125+0000", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6670000000000001e-06 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176130+0000", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.651e-06 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176134+0000", 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7680000000000002e-06 2024-09-22T01:14:53.257 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57176 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183346+0000", 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9999456, 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00054289200000000001, 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.258 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57176 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183346+0000", 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183346+0000", 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183346+0000", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3900000000000005e-07 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183348+0000", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183353+0000", 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4049999999999999e-06 2024-09-22T01:14:53.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183357+0000", 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9859999999999996e-06 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57679 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175593+0000", 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076976509999999998, 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0083224240000000001, 2024-09-22T01:14:53.261 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57679 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175593+0000", 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175593+0000", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175594+0000", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1600000000000002e-07 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175596+0000", 2024-09-22T01:14:53.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9199999999999998e-06 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175600+0000", 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8450000000000002e-06 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175604+0000", 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8190000000000002e-06 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57711 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176138+0000", 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071525820000000002, 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078044309999999997, 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57711 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176138+0000", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176138+0000", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176139+0000", 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2000000000000001e-07 2024-09-22T01:14:53.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176141+0000", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176145+0000", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7219999999999999e-06 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176149+0000", 2024-09-22T01:14:53.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8099999999999999e-06 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57177 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183395+0000", 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.999896, 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00058152699999999998, 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.268 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57177 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183395+0000", 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183395+0000", 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183395+0000", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3000000000000001e-07 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183397+0000", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0140000000000001e-06 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183403+0000", 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4e-06 2024-09-22T01:14:53.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183409+0000", 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1009999999999997e-06 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57680 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175609+0000", 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007681757, 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.008394561, 2024-09-22T01:14:53.271 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57680 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175609+0000", 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175609+0000", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175610+0000", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2099999999999998e-07 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175611+0000", 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.733e-06 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175616+0000", 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7500000000000003e-06 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175620+0000", 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7519999999999998e-06 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.274 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57712 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176154+0000", 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071373080000000002, 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0078804040000000006, 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57712 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.275 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176154+0000", 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176154+0000", 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176154+0000", 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4099999999999999e-07 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176156+0000", 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7409999999999999e-06 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176161+0000", 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.617e-06 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176164+0000", 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7840000000000001e-06 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57178 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183443+0000", 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9998474, 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00061886999999999999, 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57178 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183443+0000", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183443+0000", 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183444+0000", 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183446+0000", 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9259999999999999e-06 2024-09-22T01:14:53.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183451+0000", 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3510000000000001e-06 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183456+0000", 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3100000000000002e-06 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57681 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175648+0000", 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.007643454, 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084424070000000007, 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57681 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175648+0000", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175648+0000", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175648+0000", 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4600000000000005e-07 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175650+0000", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9649999999999998e-06 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175655+0000", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.327e-06 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175659+0000", 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8369999999999999e-06 2024-09-22T01:14:53.284 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57713 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176179+0000", 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0071118739999999998, 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0079386679999999994, 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.285 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57713 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176179+0000", 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176179+0000", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176180+0000", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4000000000000002e-07 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176182+0000", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8169999999999999e-06 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176187+0000", 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4129999999999997e-06 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176191+0000", 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0239999999999999e-06 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57179 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.288 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183487+0000", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.999804, 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00066049200000000005, 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57179 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183487+0000", 2024-09-22T01:14:53.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183487+0000", 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183488+0000", 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0800000000000005e-07 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183490+0000", 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.875e-06 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183495+0000", 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3249999999999998e-06 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183499+0000", 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0249999999999996e-06 2024-09-22T01:14:53.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57682 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175665+0000", 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076262550000000002, 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085113380000000002, 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57682 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175665+0000", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175665+0000", 2024-09-22T01:14:53.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175665+0000", 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2000000000000001e-07 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175667+0000", 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.809e-06 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175672+0000", 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8219999999999997e-06 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175676+0000", 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8240000000000001e-06 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57714 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176197+0000", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0070944909999999996, 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080079639999999994, 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57714 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176197+0000", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176197+0000", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176197+0000", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.15e-07 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176199+0000", 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.711e-06 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176204+0000", 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6350000000000003e-06 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176208+0000", 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8489999999999999e-06 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.298 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57180 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183543+0000", 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9997478, 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00068927099999999998, 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.299 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57180 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183543+0000", 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183543+0000", 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183544+0000", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9999999999999998e-07 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183546+0000", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.277e-06 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183553+0000", 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.041e-06 2024-09-22T01:14:53.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183558+0000", 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9830000000000004e-06 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57683 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.175689+0000", 2024-09-22T01:14:53.302 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0076018320000000002, 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085727890000000008, 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57683 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175689+0000", 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175689+0000", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175690+0000", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175692+0000", 2024-09-22T01:14:53.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.159e-06 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175697+0000", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5140000000000001e-06 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.175701+0000", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184266+0000", 2024-09-22T01:14:53.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085642280000000001 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57715 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176213+0000", 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0070779199999999997, 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081175689999999998, 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.306 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57715 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176213+0000", 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176213+0000", 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176213+0000", 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2300000000000002e-07 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176215+0000", 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176220+0000", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176224+0000", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8739999999999997e-06 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57181 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183579+0000", 2024-09-22T01:14:53.309 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9997125, 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000781206, 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57181 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183579+0000", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183579+0000", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183579+0000", 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183581+0000", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.933e-06 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183587+0000", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.553e-06 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183591+0000", 2024-09-22T01:14:53.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1450000000000001e-06 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57716 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176228+0000", 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0070625879999999999, 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081622220000000002, 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57716 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176228+0000", 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176228+0000", 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176229+0000", 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1300000000000001e-07 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176231+0000", 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.88e-06 2024-09-22T01:14:53.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176235+0000", 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7389999999999999e-06 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176239+0000", 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6959999999999998e-06 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57182 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183622+0000", 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9996691, 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00079709500000000005, 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57182 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183622+0000", 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183622+0000", 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183623+0000", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183625+0000", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9099999999999999e-06 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183630+0000", 2024-09-22T01:14:53.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2700000000000004e-06 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183634+0000", 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9439999999999998e-06 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57717 3.1e 3.8a409b9e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.176244+0000", 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0070472659999999999, 2024-09-22T01:14:53.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082047879999999993, 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57717 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176244+0000", 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176244+0000", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176244+0000", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1300000000000001e-07 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176246+0000", 2024-09-22T01:14:53.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.53e-06 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176250+0000", 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7140000000000001e-06 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.176254+0000", 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.726e-06 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57183 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183670+0000", 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9996204, 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00080858700000000004, 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57183 2024-09-22T01:14:53.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183670+0000", 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183670+0000", 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183671+0000", 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4700000000000001e-07 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183673+0000", 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.849e-06 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183678+0000", 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.344e-06 2024-09-22T01:14:53.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183682+0000", 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8510000000000001e-06 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57152 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182402+0000", 2024-09-22T01:14:53.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00088915700000000001, 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021047689999999998, 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57152 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182402+0000", 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182402+0000", 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182403+0000", 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4799999999999997e-07 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182407+0000", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0300000000000004e-06 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182416+0000", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.969e-06 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182423+0000", 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9659999999999999e-06 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57184 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183717+0000", 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9995737, 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000820497, 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57184 2024-09-22T01:14:53.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183717+0000", 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183717+0000", 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183718+0000", 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183719+0000", 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.725e-06 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183725+0000", 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3109999999999996e-06 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183729+0000", 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3769999999999998e-06 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57153 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182435+0000", 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.000855911, 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021320279999999998, 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57153 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182435+0000", 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182435+0000", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182436+0000", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6700000000000003e-07 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182438+0000", 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0940000000000002e-06 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182444+0000", 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7119999999999997e-06 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182448+0000", 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2740000000000001e-06 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57185 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183761+0000", 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9995298, 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00083663800000000001, 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57185 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183761+0000", 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183761+0000", 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183762+0000", 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183763+0000", 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7379999999999999e-06 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183769+0000", 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3900000000000001e-06 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183774+0000", 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4670000000000003e-06 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57154 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182456+0000", 2024-09-22T01:14:53.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00083478600000000001, 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021693099999999998, 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57154 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182456+0000", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182456+0000", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182457+0000", 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:14:53.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182459+0000", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9970000000000001e-06 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182464+0000", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182468+0000", 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0720000000000001e-06 2024-09-22T01:14:53.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57186 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183812+0000", 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9994793, 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084379799999999999, 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57186 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183812+0000", 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183812+0000", 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183812+0000", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.13e-07 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183814+0000", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.03e-06 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183820+0000", 2024-09-22T01:14:53.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3020000000000002e-06 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183824+0000", 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57155 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182486+0000", 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00080486700000000002, 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0021989380000000001, 2024-09-22T01:14:53.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57155 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182486+0000", 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182486+0000", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182487+0000", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182489+0000", 2024-09-22T01:14:53.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9429999999999999e-06 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182494+0000", 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7389999999999996e-06 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182499+0000", 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0759999999999996e-06 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57187 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183859+0000", 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9994316, 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085598300000000005, 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57187 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183859+0000", 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183859+0000", 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183860+0000", 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2799999999999996e-07 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183862+0000", 2024-09-22T01:14:53.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.857e-06 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183867+0000", 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3349999999999997e-06 2024-09-22T01:14:53.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183872+0000", 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.673e-06 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57156 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182527+0000", 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00076451899999999996, 2024-09-22T01:14:53.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002216689, 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57156 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182527+0000", 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182527+0000", 2024-09-22T01:14:53.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182527+0000", 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5300000000000004e-07 2024-09-22T01:14:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182539+0000", 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2312e-05 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182548+0000", 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.157e-06 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182552+0000", 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2710000000000003e-06 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57188 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183904+0000", 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9993868, 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00086957799999999997, 2024-09-22T01:14:53.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57188 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183904+0000", 2024-09-22T01:14:53.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183904+0000", 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183905+0000", 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183907+0000", 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8560000000000001e-06 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183912+0000", 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3249999999999998e-06 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183916+0000", 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9600000000000002e-06 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57157 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182573+0000", 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00071837699999999997, 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0022288809999999998, 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57157 2024-09-22T01:14:53.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182573+0000", 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182573+0000", 2024-09-22T01:14:53.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182573+0000", 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6100000000000001e-07 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182575+0000", 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9889999999999999e-06 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182581+0000", 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8769999999999998e-06 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182585+0000", 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.766e-06 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57189 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183939+0000", 2024-09-22T01:14:53.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9993525, 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00089425699999999997, 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57189 2024-09-22T01:14:53.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183939+0000", 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183939+0000", 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183939+0000", 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3700000000000001e-07 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183941+0000", 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.844e-06 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183949+0000", 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9549999999999992e-06 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183956+0000", 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6660000000000002e-06 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57158 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182591+0000", 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00070047999999999996, 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002270163, 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.385 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57158 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182591+0000", 2024-09-22T01:14:53.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182591+0000", 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182591+0000", 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.46e-07 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182593+0000", 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182598+0000", 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8790000000000002e-06 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182601+0000", 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.754e-06 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57190 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183964+0000", 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9993267, 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00092723299999999996, 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.389 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57190 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183964+0000", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183964+0000", 2024-09-22T01:14:53.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183965+0000", 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183968+0000", 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8289999999999999e-06 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183976+0000", 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3129999999999999e-06 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183983+0000", 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4590000000000003e-06 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57159 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182620+0000", 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00067082800000000005, 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023007679999999999, 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57159 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182620+0000", 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182620+0000", 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182621+0000", 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1699999999999998e-07 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182623+0000", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8279999999999999e-06 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182628+0000", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4410000000000001e-06 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182633+0000", 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.673e-06 2024-09-22T01:14:53.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57191 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183991+0000", 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9993, 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000958897, 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57191 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183991+0000", 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183991+0000", 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183992+0000", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8699999999999995e-07 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184006+0000", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4701999999999999e-05 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184016+0000", 2024-09-22T01:14:53.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7750000000000004e-06 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184025+0000", 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7579999999999998e-06 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57160 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182660+0000", 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00063111099999999996, 2024-09-22T01:14:53.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023206440000000002, 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57160 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182660+0000", 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182660+0000", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182660+0000", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182662+0000", 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8139999999999999e-06 2024-09-22T01:14:53.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182668+0000", 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.463e-06 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182673+0000", 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8740000000000003e-06 2024-09-22T01:14:53.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57192 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184033+0000", 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.999258, 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000975829, 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57192 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184033+0000", 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184033+0000", 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184034+0000", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7399999999999998e-07 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184036+0000", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1229999999999998e-06 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184041+0000", 2024-09-22T01:14:53.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3859999999999998e-06 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184046+0000", 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.746e-06 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57161 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182680+0000", 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00061107299999999995, 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023594839999999998, 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57161 2024-09-22T01:14:53.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182680+0000", 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182680+0000", 2024-09-22T01:14:53.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182680+0000", 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9999999999999998e-07 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182682+0000", 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7549999999999999e-06 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182687+0000", 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8940000000000001e-06 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182691+0000", 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0659999999999997e-06 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57193 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184051+0000", 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9992404, 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001017111, 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57193 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184051+0000", 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184051+0000", 2024-09-22T01:14:53.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184051+0000", 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184053+0000", 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7179999999999999e-06 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184058+0000", 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8439999999999997e-06 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184062+0000", 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6990000000000001e-06 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57162 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182708+0000", 2024-09-22T01:14:53.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00058309900000000001, 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0023909410000000002, 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57162 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182708+0000", 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182708+0000", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182708+0000", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0999999999999999e-07 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182710+0000", 2024-09-22T01:14:53.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182716+0000", 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4990000000000002e-06 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182720+0000", 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8929999999999998e-06 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57194 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184067+0000", 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9992242, 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001060872, 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57194 2024-09-22T01:14:53.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184067+0000", 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184067+0000", 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184067+0000", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184069+0000", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7940000000000001e-06 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184074+0000", 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7979999999999996e-06 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184077+0000", 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6380000000000002e-06 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57163 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182749+0000", 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00054239700000000002, 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002409408, 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57163 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182749+0000", 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182749+0000", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182749+0000", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2099999999999997e-07 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182751+0000", 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.933e-06 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182757+0000", 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4729999999999999e-06 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182760+0000", 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8720000000000004e-06 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57195 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184082+0000", 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9992089, 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001104437, 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57195 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184082+0000", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184082+0000", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184082+0000", 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2599999999999998e-07 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184084+0000", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6020000000000001e-06 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184089+0000", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6500000000000004e-06 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184093+0000", 2024-09-22T01:14:53.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5240000000000002e-06 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57164 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182789+0000", 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.000501872, 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024277499999999998, 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57164 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182789+0000", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182789+0000", 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182790+0000", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182792+0000", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9199999999999998e-06 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182797+0000", 2024-09-22T01:14:53.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4280000000000004e-06 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182801+0000", 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9110000000000003e-06 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57196 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184123+0000", 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9991684, 2024-09-22T01:14:53.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011229720000000001, 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57196 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184123+0000", 2024-09-22T01:14:53.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184123+0000", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184123+0000", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1799999999999995e-07 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184125+0000", 2024-09-22T01:14:53.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9920000000000002e-06 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184131+0000", 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7910000000000001e-06 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184135+0000", 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.755e-06 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57165 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182833+0000", 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00045843000000000001, 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0024435440000000002, 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57165 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182833+0000", 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182833+0000", 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182833+0000", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1200000000000003e-07 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182835+0000", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.866e-06 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182840+0000", 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3669999999999996e-06 2024-09-22T01:14:53.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182844+0000", 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8680000000000001e-06 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57197 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184173+0000", 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9991183, 2024-09-22T01:14:53.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001131782, 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57197 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184173+0000", 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184173+0000", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184173+0000", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0500000000000004e-07 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184175+0000", 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184181+0000", 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5040000000000002e-06 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184185+0000", 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57166 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182883+0000", 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00040807000000000001, 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025185749999999999, 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57166 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182883+0000", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182883+0000", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182883+0000", 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9699999999999996e-07 2024-09-22T01:14:53.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182885+0000", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.888e-06 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182891+0000", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182895+0000", 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8850000000000001e-06 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57198 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184222+0000", 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9990692, 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0012092610000000001, 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57198 2024-09-22T01:14:53.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184222+0000", 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184222+0000", 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184222+0000", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2900000000000004e-07 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184224+0000", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7969999999999999e-06 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184230+0000", 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2650000000000004e-06 2024-09-22T01:14:53.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184233+0000", 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8539999999999999e-06 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57167 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182933+0000", 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00035851200000000001, 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025304920000000001, 2024-09-22T01:14:53.452 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57167 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182933+0000", 2024-09-22T01:14:53.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182933+0000", 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182933+0000", 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1200000000000003e-07 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182935+0000", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.702e-06 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182940+0000", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3990000000000003e-06 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182944+0000", 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8249999999999998e-06 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57199 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184262+0000", 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9990292, 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001229532, 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57199 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184262+0000", 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184262+0000", 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184263+0000", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184264+0000", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8020000000000001e-06 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184270+0000", 2024-09-22T01:14:53.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184275+0000", 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.339e-06 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57168 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.182983+0000", 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00030811, 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025387470000000001, 2024-09-22T01:14:53.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57168 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182983+0000", 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182983+0000", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182983+0000", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1799999999999995e-07 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.182994+0000", 2024-09-22T01:14:53.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0668e-05 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183000+0000", 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.268e-06 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183005+0000", 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1489999999999996e-06 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.462 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57395 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184286+0000", 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9990048, 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001265841, 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57395 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184286+0000", 2024-09-22T01:14:53.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184286+0000", 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184287+0000", 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3500000000000002e-07 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184288+0000", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8339999999999999e-06 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184294+0000", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1610000000000002e-06 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184297+0000", 2024-09-22T01:14:53.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8e-06 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57169 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.183031+0000", 2024-09-22T01:14:53.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.000259936, 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025498589999999998, 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57169 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183031+0000", 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183031+0000", 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183032+0000", 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1900000000000003e-07 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183034+0000", 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0080000000000001e-06 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183039+0000", 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6640000000000003e-06 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.183043+0000", 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8430000000000003e-06 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:57396 3.1e 3.8a409b9e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:53.184303+0000", 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4294967295.9989882, 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001308755, 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "queued for pg", 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 57396 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184303+0000", 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184303+0000", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184303+0000", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2799999999999996e-07 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184305+0000", 2024-09-22T01:14:53.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.823e-06 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184310+0000", 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6770000000000003e-06 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:53.184313+0000", 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.681e-06 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-22T01:14:53.476 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 76 2024-09-22T01:14:53.476 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:53.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:53.478 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:53.475+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:53.577 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:53.574+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:53.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:14:53.664 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:53.664 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:53.665 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:53.665 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:53.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:53.704 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:53.702+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:53.778 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:53.775+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51347 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3348480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843289+0000", 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96270621199999995, 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051870800000000002, 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51347 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843289+0000", 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843289+0000", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843290+0000", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6099999999999999e-07 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843296+0000", 2024-09-22T01:14:53.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.849e-06 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843309+0000", 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2439999999999999e-05 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843316+0000", 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4619999999999999e-06 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.894917+0000", 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051600910999999999 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895027+0000", 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109425 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895159+0000", 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013275200000000001 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51349 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3356672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843329+0000", 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96266584899999996, 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052108860999999999, 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51349 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.813 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843329+0000", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843329+0000", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843330+0000", 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0200000000000002e-07 2024-09-22T01:14:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843333+0000", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2789999999999999e-06 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843345+0000", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2136999999999999e-05 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843375+0000", 2024-09-22T01:14:53.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9878000000000001e-05 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895200+0000", 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051825126999999999 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895308+0000", 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010812 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895438+0000", 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012971799999999999 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51351 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3364864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843393+0000", 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96260146300000005, 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052510767999999999, 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51351 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843393+0000", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843393+0000", 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843394+0000", 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1200000000000002e-07 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843397+0000", 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.388e-06 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843406+0000", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5709999999999998e-06 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843412+0000", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3910000000000003e-06 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895501+0000", 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052088846000000001 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895685+0000", 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000183296 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895904+0000", 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000219564 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51353 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3373056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843425+0000", 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96256998199999999, 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052960621999999999, 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51353 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843425+0000", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843425+0000", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843425+0000", 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3600000000000003e-07 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843429+0000", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0730000000000001e-06 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843439+0000", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0705e-05 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843445+0000", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8080000000000001e-06 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.895976+0000", 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052531147 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896161+0000", 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00018527499999999999 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896385+0000", 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223978 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51355 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3381248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843456+0000", 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96253846200000004, 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053225762000000003, 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51355 2024-09-22T01:14:53.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843456+0000", 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843456+0000", 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843457+0000", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1500000000000005e-07 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843460+0000", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1080000000000001e-06 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843468+0000", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5569999999999998e-06 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843474+0000", 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3709999999999996e-06 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896432+0000", 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052958516999999997 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896545+0000", 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011254199999999999 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896682+0000", 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000137152 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51371 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3389440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843487+0000", 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96250758300000006, 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053472912999999997, 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.831 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51371 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843487+0000", 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843487+0000", 2024-09-22T01:14:53.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843488+0000", 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.46e-07 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843490+0000", 2024-09-22T01:14:53.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5469999999999999e-06 2024-09-22T01:14:53.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843498+0000", 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4669999999999998e-06 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843515+0000", 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6915999999999999e-05 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896723+0000", 2024-09-22T01:14:53.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053208790999999998 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896832+0000", 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108305 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896960+0000", 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000128441 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51373 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3397632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843579+0000", 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96241611100000002, 2024-09-22T01:14:53.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053652391000000001, 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.841 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51373 2024-09-22T01:14:53.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843579+0000", 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843579+0000", 2024-09-22T01:14:53.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843579+0000", 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0599999999999999e-07 2024-09-22T01:14:53.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843583+0000", 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6320000000000001e-06 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843596+0000", 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2989e-05 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843621+0000", 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4660000000000001e-05 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.896999+0000", 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053378111999999998 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.897106+0000", 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107045 2024-09-22T01:14:53.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.897231+0000", 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012514700000000001 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.849 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51375 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3405824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.843680+0000", 2024-09-22T01:14:53.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.96231496699999997, 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053851501000000003, 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51375 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843680+0000", 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843680+0000", 2024-09-22T01:14:53.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843681+0000", 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8899999999999998e-07 2024-09-22T01:14:53.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843684+0000", 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7620000000000001e-06 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843695+0000", 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0774999999999999e-05 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.843702+0000", 2024-09-22T01:14:53.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4919999999999998e-06 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.897272+0000", 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053570155000000001 2024-09-22T01:14:53.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.897381+0000", 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109131 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.897531+0000", 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000150397 2024-09-22T01:14:53.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51653 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3702784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856185+0000", 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94980937600000004, 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052106305999999998, 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51653 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856185+0000", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856185+0000", 2024-09-22T01:14:53.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856186+0000", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.58e-07 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856188+0000", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4399999999999999e-06 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856195+0000", 2024-09-22T01:14:53.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4779999999999996e-06 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856200+0000", 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8570000000000003e-06 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908082+0000", 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051882648000000003 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908177+0000", 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4932999999999994e-05 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908292+0000", 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114592 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51655 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3710976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856206+0000", 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94978867300000003, 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052343539000000001, 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51655 2024-09-22T01:14:53.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856206+0000", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856206+0000", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856206+0000", 2024-09-22T01:14:53.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0400000000000002e-07 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856209+0000", 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856217+0000", 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6279999999999994e-06 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856222+0000", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7990000000000001e-06 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908325+0000", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052102884000000002 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908421+0000", 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5463000000000004e-05 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908550+0000", 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012901899999999999 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51657 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3719168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856235+0000", 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94975989900000002, 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052455648000000001, 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.868 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51657 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856235+0000", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856235+0000", 2024-09-22T01:14:53.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856235+0000", 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856238+0000", 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3609999999999999e-06 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856245+0000", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.131e-06 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856250+0000", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9439999999999996e-06 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908578+0000", 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052328147999999998 2024-09-22T01:14:53.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908630+0000", 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1907999999999998e-05 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908691+0000", 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0754000000000001e-05 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51659 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3727360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856274+0000", 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94972060599999997, 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052580731999999998, 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51659 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856274+0000", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856274+0000", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856275+0000", 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.61e-07 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856278+0000", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7980000000000001e-06 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856285+0000", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.79e-06 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856291+0000", 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:14:53.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908721+0000", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430350000000001 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908776+0000", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4718e-05 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908855+0000", 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9085999999999995e-05 2024-09-22T01:14:53.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51661 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3735552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856299+0000", 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94969614400000002, 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052687291999999997, 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.877 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51661 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856299+0000", 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856299+0000", 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856299+0000", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7199999999999999e-07 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856302+0000", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5519999999999999e-06 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856309+0000", 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:14:53.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856317+0000", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5420000000000005e-06 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908876+0000", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052558692999999997 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908926+0000", 2024-09-22T01:14:53.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0389000000000002e-05 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.908986+0000", 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9762000000000001e-05 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51663 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3743744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856325+0000", 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94966997600000003, 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052898355000000001, 2024-09-22T01:14:53.881 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51663 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856325+0000", 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856325+0000", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856325+0000", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856332+0000", 2024-09-22T01:14:53.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1590000000000004e-06 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856340+0000", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.114e-06 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856344+0000", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9949999999999996e-06 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909030+0000", 2024-09-22T01:14:53.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052685787999999997 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909121+0000", 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0851000000000001e-05 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909223+0000", 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102043 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.885 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51664 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3751936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856354+0000", 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94964098600000002, 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053019636000000002, 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.886 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51664 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856354+0000", 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856354+0000", 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856354+0000", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1899999999999998e-07 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856357+0000", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4540000000000001e-06 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856363+0000", 2024-09-22T01:14:53.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6109999999999999e-06 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856376+0000", 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3072e-05 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909254+0000", 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052877133 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909304+0000", 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0228000000000001e-05 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909373+0000", 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9719000000000001e-05 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51666 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3760128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.890 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856387+0000", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94960751399999999, 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053237387999999997, 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51666 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856387+0000", 2024-09-22T01:14:53.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856387+0000", 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856388+0000", 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7999999999999996e-07 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856391+0000", 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.914e-06 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856398+0000", 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0430000000000002e-06 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856408+0000", 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0661e-05 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909428+0000", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053019230000000001 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909522+0000", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4264000000000004e-05 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909625+0000", 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000102796 2024-09-22T01:14:53.894 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51667 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3768320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856430+0000", 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94956507999999995, 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05333947, 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51667 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856430+0000", 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856430+0000", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856430+0000", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856433+0000", 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.004e-06 2024-09-22T01:14:53.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856442+0000", 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.191e-06 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856447+0000", 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3530000000000002e-06 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909653+0000", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053206003000000002 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909705+0000", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1647000000000002e-05 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909769+0000", 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4610000000000007e-05 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.899 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51668 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3776512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856456+0000", 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94953928300000001, 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053510437000000001, 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.900 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51668 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856456+0000", 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856456+0000", 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856456+0000", 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2100000000000002e-07 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856458+0000", 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.548e-06 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856465+0000", 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5819999999999998e-06 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856473+0000", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6879999999999999e-06 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909797+0000", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053324725000000003 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909858+0000", 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0504000000000002e-05 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.909966+0000", 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107969 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51669 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3784704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856480+0000", 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94951468100000003, 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053642677, 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51669 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856480+0000", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856480+0000", 2024-09-22T01:14:53.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856481+0000", 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8299999999999997e-07 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856483+0000", 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4530000000000001e-06 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856490+0000", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.669e-06 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856495+0000", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910007+0000", 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053511717 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910060+0000", 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3137000000000003e-05 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910123+0000", 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3002000000000004e-05 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:51670 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 3792896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:14:53.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:52.856501+0000", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.94949335999999995, 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053894751999999997, 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 51670 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856501+0000", 2024-09-22T01:14:53.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856501+0000", 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856502+0000", 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5400000000000002e-07 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:53.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856504+0000", 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6469999999999998e-06 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856513+0000", 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6729999999999998e-06 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.856524+0000", 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1110999999999999e-05 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910164+0000", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053639892000000002 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910258+0000", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4382000000000005e-05 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:52.910396+0000", 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013769300000000001 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:53.913 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:53.913 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:53.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:53.915 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:53.910+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:54.015 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:54.010+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:54.089 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:54.089 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:54.103 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:54.149 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:54.147+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:54.217 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:54.211+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:54.229 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:54.229 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:54.230 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:54.230 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:54.244 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:54.329 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:54.327+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:54.405 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:54.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.091251108000002, 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:14:54.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:14:54.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:54.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.091198888000001, 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:14:54.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090999240000002, 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:14:54.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090920537000002, 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:14:54.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:14:54.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090860544000002, 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:14:54.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:14:54.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.428 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090788050999997, 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:14:54.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:14:54.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:14:54.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.434 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090719739000001, 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.435 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:14:54.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090661814999997, 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.440 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:14:54.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:14:54.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 38.090605926000002, 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:54.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:14:54.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.203661148999998, 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:14:54.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:14:54.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.508294168999999, 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:54.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:14:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.116169158000002, 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:14:54.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.42431491, 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:14:54.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:14:54.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:14:54.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.424196690999999, 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:14:54.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:14:54.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.475 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.42166975, 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:14:54.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:14:54.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:14:54.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.403223397000001, 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.482 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:14:54.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.487 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.243596503999999, 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.488 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:14:54.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.243542409, 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:14:54.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.499 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.243456113000001, 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.500 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:14:54.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.242826403999999, 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:14:54.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:14:54.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:14:54.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:54.511 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:54.514 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:54.514 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:54.511+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:54.514 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-22T01:14:54.514 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2223 write 0x9381f8a thru 0x938c684 (0xa6fb bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2224 read 0xca33029 thru 0xca3849d (0x5475 bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2225 write 0x7a49932 thru 0x7a574f9 (0xdbc8 bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2226 read 0x44d95ba thru 0x44db7f3 (0x223a bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2228 write 0x20e241a thru 0x20ee21c (0xbe03 bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2230 read 0xd13d37f thru 0xd14582f (0x84b1 bytes) 2024-09-22T01:14:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:2231 writesame 0x773c242 thru 0x773fc45 data_size 0x2f(0x3a04 bytes) 2024-09-22T01:14:54.523 INFO:teuthology.orchestra.run.smithi096.stdout:2232 punch from 0x7ff6263 to 0x8002728, (0xc4c5 bytes) 2024-09-22T01:14:54.523 INFO:teuthology.orchestra.run.smithi096.stdout:2234 punch from 0x48b80d5 to 0x48c7250, (0xf17b bytes) 2024-09-22T01:14:54.523 INFO:teuthology.orchestra.run.smithi096.stdout:2235 write 0x910886 thru 0x91cdbc (0xc537 bytes) 2024-09-22T01:14:54.600 INFO:teuthology.orchestra.run.smithi096.stdout:2236 trunc from 0xdd04e4a to 0x7ec1540 2024-09-22T01:14:54.614 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:54.611+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:54.636 INFO:teuthology.orchestra.run.smithi096.stdout:2237 read 0x4a948af thru 0x4a997b9 (0x4f0b bytes) 2024-09-22T01:14:54.636 INFO:teuthology.orchestra.run.smithi096.stdout:2238 read 0x43550d1 thru 0x436087f (0xb7af bytes) 2024-09-22T01:14:54.645 INFO:teuthology.orchestra.run.smithi096.stdout:2239 write 0x854b18e thru 0x8559256 (0xe0c9 bytes) 2024-09-22T01:14:54.648 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:14:54.654 INFO:teuthology.orchestra.run.smithi096.stdout:2240 read 0x4bddce2 thru 0x4be83b6 (0xa6d5 bytes) 2024-09-22T01:14:54.654 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.654 INFO:teuthology.orchestra.run.smithi096.stdout:2242 read 0x6aba52d thru 0x6ac572c (0xb200 bytes) 2024-09-22T01:14:54.657 INFO:teuthology.orchestra.run.smithi096.stdout:2243 trunc from 0x8559257 to 0x9afc300 2024-09-22T01:14:54.666 INFO:teuthology.orchestra.run.smithi096.stdout:2244 read 0x7fad9fa thru 0x7fb09c3 (0x2fca bytes) 2024-09-22T01:14:54.667 INFO:teuthology.orchestra.run.smithi096.stdout:2246 read 0x49ec705 thru 0x49fbcb1 (0xf5ad bytes) 2024-09-22T01:14:54.667 INFO:teuthology.orchestra.run.smithi096.stdout:2247 read 0x247d344 thru 0x248c088 (0xed45 bytes) 2024-09-22T01:14:54.667 INFO:teuthology.orchestra.run.smithi096.stdout:2248 read 0x378ef6c thru 0x379e7d2 (0xf867 bytes) 2024-09-22T01:14:54.667 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.667 INFO:teuthology.orchestra.run.smithi096.stdout:2250 writesame 0x692cfbb thru 0x692eb90 data_size 0x7(0x1bd6 bytes) 2024-09-22T01:14:54.677 INFO:teuthology.orchestra.run.smithi096.stdout:2251 trunc from 0x9afc300 to 0x6eef0dc 2024-09-22T01:14:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:54.692 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:54.692 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:54.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:54.715 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:54.710+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:54.718 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:54.734 INFO:teuthology.orchestra.run.smithi096.stdout:2255 writesame 0xe32f409 thru 0xe332c76 data_size 0x1f(0x386e bytes) 2024-09-22T01:14:54.738 INFO:teuthology.orchestra.run.smithi096.stdout:2256 trunc from 0xe332c77 to 0xb9d5e06 2024-09-22T01:14:54.744 INFO:teuthology.orchestra.run.smithi096.stdout:2261 write 0x39acffd thru 0x39ad0b1 (0xb5 bytes) 2024-09-22T01:14:54.749 INFO:teuthology.orchestra.run.smithi096.stdout:2262 write 0x8b87506 thru 0x8b89702 (0x21fd bytes) 2024-09-22T01:14:54.753 INFO:teuthology.orchestra.run.smithi096.stdout:2264 punch from 0x984f68 to 0x986209, (0x12a1 bytes) 2024-09-22T01:14:54.753 INFO:teuthology.orchestra.run.smithi096.stdout:2265 read 0x560f5ac thru 0x56193dc (0x9e31 bytes) 2024-09-22T01:14:54.754 INFO:teuthology.orchestra.run.smithi096.stdout:2270 punch from 0x9f3a6e8 to 0x9f47237, (0xcb4f bytes) 2024-09-22T01:14:54.754 INFO:teuthology.orchestra.run.smithi096.stdout:2271 writesame 0x7011a4b thru 0x7019ccc data_size 0x41(0x8282 bytes) 2024-09-22T01:14:54.762 INFO:teuthology.orchestra.run.smithi096.stdout:2272 trunc from 0xb9d5e06 to 0x8aeb874 2024-09-22T01:14:54.782 INFO:teuthology.orchestra.run.smithi096.stdout:2274 trunc from 0x8aeb874 to 0x4ace70 2024-09-22T01:14:54.816 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:54.811+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:54.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:54.904 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:54.904 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:54.904 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:54.917 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:54.914+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:54.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:55.017 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:55.015+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.853314271999999, 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:14:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.853239443, 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:14:55.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:14:55.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:55.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.853201419000001, 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:55.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:14:55.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:14:55.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:55.101 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.853152197, 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:14:55.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:14:55.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:55.107 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.853115850000002, 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:14:55.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.19424296, 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.113 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.192062214, 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:55.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:14:55.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.191989728999999, 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.122 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:14:55.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:55.126 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.191927912000001, 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.129 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.130 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.788760332999999, 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:55.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.112078552, 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:14:55.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.112022927000002, 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:14:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:14:55.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111920306999998, 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:14:55.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:14:55.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111819819000001, 2024-09-22T01:14:55.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:14:55.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:14:55.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111759251999999, 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.153 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:14:55.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111701877000002, 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:14:55.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111633167000001, 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:14:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:14:55.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111571222999999, 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:14:55.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.170 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.111507255999999, 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:55.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:14:55.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 11.255412116, 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:14:55.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:14:55.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:55.179 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:55.180 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:55.181 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:55.181 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:55.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:55.182 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:55.182 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:55.178+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:55.186 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:55.186 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:55.244 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:55.244 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:55.279 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:55.280 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:55.281 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:55.278+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:55.317 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:55.317 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:55.342 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:55.343 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:55.346 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:55.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:55.380 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:55.380 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:55.382 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:55.378+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:55.388 INFO:teuthology.orchestra.run.smithi096.stdout:2277 punch from 0x480eb7 to 0x48c57a, (0xb6c3 bytes) 2024-09-22T01:14:55.400 INFO:teuthology.orchestra.run.smithi096.stdout:2278 writesame 0x35b569b thru 0x35bd9c7 data_size 0x1(0x832d bytes) 2024-09-22T01:14:55.407 INFO:teuthology.orchestra.run.smithi096.stdout:2281 trunc from 0x35bd9c8 to 0x9514772 2024-09-22T01:14:55.414 INFO:teuthology.orchestra.run.smithi096.stdout:2282 read 0x359dfc8 thru 0x35a6cc5 (0x8cfe bytes) 2024-09-22T01:14:55.419 INFO:teuthology.orchestra.run.smithi096.stdout:2283 trunc from 0x9514772 to 0x5521b5b 2024-09-22T01:14:55.425 INFO:teuthology.orchestra.run.smithi096.stdout:2284 punch from 0x3d9f52b to 0x3da393f, (0x4414 bytes) 2024-09-22T01:14:55.443 INFO:teuthology.orchestra.run.smithi096.stdout:2286 writesame 0xe523906 thru 0xe53168b data_size 0x35(0xdd86 bytes) 2024-09-22T01:14:55.444 INFO:teuthology.orchestra.run.smithi096.stdout:2287 writesame 0x4af847 thru 0x4b4a44 data_size 0x1(0x51fe bytes) 2024-09-22T01:14:55.469 INFO:teuthology.orchestra.run.smithi096.stdout:2288 trunc from 0xe53168c to 0x76c20c3 2024-09-22T01:14:55.477 INFO:teuthology.orchestra.run.smithi096.stdout:2290 punch from 0x1aeeeaa to 0x1afb768, (0xc8be bytes) 2024-09-22T01:14:55.482 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:55.478+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:55.492 INFO:teuthology.orchestra.run.smithi096.stdout:2291 writesame 0xd0876ca thru 0xd095899 data_size 0x8(0xe1d0 bytes) 2024-09-22T01:14:55.492 INFO:teuthology.orchestra.run.smithi096.stdout:2292 read 0x3d28427 thru 0x3d2c6ee (0x42c8 bytes) 2024-09-22T01:14:55.502 INFO:teuthology.orchestra.run.smithi096.stdout:2293 write 0xd4f6b51 thru 0xd503d11 (0xd1c1 bytes) 2024-09-22T01:14:55.509 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:55.509 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:55.510 INFO:teuthology.orchestra.run.smithi096.stdout:2294 punch from 0xb77cd15 to 0xb77eaec, (0x1dd7 bytes) 2024-09-22T01:14:55.521 INFO:teuthology.orchestra.run.smithi096.stdout:2299 write 0xdaa3695 thru 0xdab365d (0xffc9 bytes) 2024-09-22T01:14:55.525 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:55.530 INFO:teuthology.orchestra.run.smithi096.stdout:2300 punch from 0x4327359 to 0x43352bf, (0xdf66 bytes) 2024-09-22T01:14:55.530 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.539 INFO:teuthology.orchestra.run.smithi096.stdout:2305 write 0xeb33f10 thru 0xeb3d914 (0x9a05 bytes) 2024-09-22T01:14:55.545 INFO:teuthology.orchestra.run.smithi096.stdout:2306 punch from 0xb96ff92 to 0xb97e2e7, (0xe355 bytes) 2024-09-22T01:14:55.545 INFO:teuthology.orchestra.run.smithi096.stdout:2307 write 0x7d737c thru 0x7dc8cc (0x5551 bytes) 2024-09-22T01:14:55.555 INFO:teuthology.orchestra.run.smithi096.stdout:2308 trunc from 0xeb3d915 to 0xa016c66 2024-09-22T01:14:55.569 INFO:teuthology.orchestra.run.smithi096.stdout:2309 write 0x9f79c5f thru 0x9f82798 (0x8b3a bytes) 2024-09-22T01:14:55.584 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:55.579+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:55.586 INFO:teuthology.orchestra.run.smithi096.stdout:2310 write 0xbe27a77 thru 0xbe2ceab (0x5435 bytes) 2024-09-22T01:14:55.589 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.589 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.589 INFO:teuthology.orchestra.run.smithi096.stdout:2315 writesame 0xa261592 thru 0xa266512 data_size 0x1(0x4f81 bytes) 2024-09-22T01:14:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:2316 write 0x912e54c thru 0x9135380 (0x6e35 bytes) 2024-09-22T01:14:55.614 INFO:teuthology.orchestra.run.smithi096.stdout:2317 punch from 0x15293ea to 0x1531fea, (0x8c00 bytes) 2024-09-22T01:14:55.625 INFO:teuthology.orchestra.run.smithi096.stdout:2321 writesame 0xea50b22 thru 0xea5b36d data_size 0x2(0xa84c bytes) 2024-09-22T01:14:55.625 INFO:teuthology.orchestra.run.smithi096.stdout:2322 punch from 0x172c407 to 0x17359a6, (0x959f bytes) 2024-09-22T01:14:55.625 INFO:teuthology.orchestra.run.smithi096.stdout:2323 write 0x9994b65 thru 0x999b77b (0x6c17 bytes) 2024-09-22T01:14:55.628 INFO:teuthology.orchestra.run.smithi096.stdout:2324 write 0x882f440 thru 0x883a8f9 (0xb4ba bytes) 2024-09-22T01:14:55.638 INFO:teuthology.orchestra.run.smithi096.stdout:2326 read 0x485e045 thru 0x486c218 (0xe1d4 bytes) 2024-09-22T01:14:55.638 INFO:teuthology.orchestra.run.smithi096.stdout:2327 read 0x7eb5b4d thru 0x7ebf1a3 (0x9657 bytes) 2024-09-22T01:14:55.638 INFO:teuthology.orchestra.run.smithi096.stdout:2329 read 0xd902c06 thru 0xd911a51 (0xee4c bytes) 2024-09-22T01:14:55.638 INFO:teuthology.orchestra.run.smithi096.stdout:2330 read 0x9513bfb thru 0x951738c (0x3792 bytes) 2024-09-22T01:14:55.638 INFO:teuthology.orchestra.run.smithi096.stdout:2331 writesame 0xb398862 thru 0xb3a7fd0 data_size 0x1(0xf76f bytes) 2024-09-22T01:14:55.657 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:55.657 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:55.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:14:55.714 INFO:teuthology.orchestra.run.smithi096.stdout:2334 punch from 0x1f4d4f7 to 0x1f5090f, (0x3418 bytes) 2024-09-22T01:14:55.714 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.714 INFO:teuthology.orchestra.run.smithi096.stdout:2336 write 0x638e91f thru 0x638f174 (0x856 bytes) 2024-09-22T01:14:55.715 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:55.710+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:55.722 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.722 INFO:teuthology.orchestra.run.smithi096.stdout:2339 write 0x36d7236 thru 0x36d7432 (0x1fd bytes) 2024-09-22T01:14:55.728 INFO:teuthology.orchestra.run.smithi096.stdout:2340 trunc from 0xea5b36e to 0xc2e24bd 2024-09-22T01:14:55.742 INFO:teuthology.orchestra.run.smithi096.stdout:2341 read 0x79e1099 thru 0x79e3ca4 (0x2c0c bytes) 2024-09-22T01:14:55.742 INFO:teuthology.orchestra.run.smithi096.stdout:2343 punch from 0x2cd18c to 0x2cf580, (0x23f4 bytes) 2024-09-22T01:14:55.754 INFO:teuthology.orchestra.run.smithi096.stdout:2344 trunc from 0xc2e24bd to 0x367ab4f 2024-09-22T01:14:55.772 INFO:teuthology.orchestra.run.smithi096.stdout:2345 flatten 2024-09-22T01:14:55.785 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:55.782+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:55.799 INFO:teuthology.orchestra.run.smithi096.stdout:2346 write 0xa40a4ea thru 0xa41953a (0xf051 bytes) 2024-09-22T01:14:55.807 INFO:teuthology.orchestra.run.smithi096.stdout:2348 read 0x4aa6a07 thru 0x4aaad7f (0x4379 bytes) 2024-09-22T01:14:55.808 INFO:teuthology.orchestra.run.smithi096.stdout:2349 read 0x9c16044 thru 0x9c227fc (0xc7b9 bytes) 2024-09-22T01:14:55.808 INFO:teuthology.orchestra.run.smithi096.stdout:2350 punch from 0x377d36 to 0x37943f, (0x1709 bytes) 2024-09-22T01:14:55.808 INFO:teuthology.orchestra.run.smithi096.stdout:2351 writesame 0x2e719fd thru 0x2e7b16c data_size 0x8(0x9770 bytes) 2024-09-22T01:14:55.814 INFO:teuthology.orchestra.run.smithi096.stdout:2352 read 0x81083bf thru 0x8116702 (0xe344 bytes) 2024-09-22T01:14:55.814 INFO:teuthology.orchestra.run.smithi096.stdout:2353 read 0x53a7233 thru 0x53b002b (0x8df9 bytes) 2024-09-22T01:14:55.814 INFO:teuthology.orchestra.run.smithi096.stdout:2354 writesame 0xc0118a thru 0xc03638 data_size 0x1(0x24af bytes) 2024-09-22T01:14:55.824 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.824 INFO:teuthology.orchestra.run.smithi096.stdout:2357 write 0x4ae3fc3 thru 0x4ae6a59 (0x2a97 bytes) 2024-09-22T01:14:55.830 INFO:teuthology.orchestra.run.smithi096.stdout:2359 punch from 0x403e6d6 to 0x404923d, (0xab67 bytes) 2024-09-22T01:14:55.830 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.834 INFO:teuthology.orchestra.run.smithi096.stdout:2362 trunc from 0xa41953b to 0x61e4111 2024-09-22T01:14:55.842 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:55.842 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:55.851 INFO:teuthology.orchestra.run.smithi096.stdout:2363 read 0x543e40f thru 0x5441b20 (0x3712 bytes) 2024-09-22T01:14:55.852 INFO:teuthology.orchestra.run.smithi096.stdout:2364 trunc from 0x61e4111 to 0x4e76e98 2024-09-22T01:14:55.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:14:55.861 INFO:teuthology.orchestra.run.smithi096.stdout:2366 read 0x313d6d4 thru 0x314c079 (0xe9a6 bytes) 2024-09-22T01:14:55.861 INFO:teuthology.orchestra.run.smithi096.stdout:2367 writesame 0x1d441fe thru 0x1d4f19e data_size 0x7(0xafa1 bytes) 2024-09-22T01:14:55.887 INFO:teuthology.orchestra.run.smithi096.stdout:2369 write 0xb94f3a9 thru 0xb94f623 (0x27b bytes) 2024-09-22T01:14:55.887 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:55.882+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:55.891 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.891 INFO:teuthology.orchestra.run.smithi096.stdout:2371 read 0x427a7fe thru 0x427e71e (0x3f21 bytes) 2024-09-22T01:14:55.902 INFO:teuthology.orchestra.run.smithi096.stdout:2372 writesame 0xe145692 thru 0xe14edb0 data_size 0x1(0x971f bytes) 2024-09-22T01:14:55.902 INFO:teuthology.orchestra.run.smithi096.stdout:2373 read 0x5215698 thru 0x522240a (0xcd73 bytes) 2024-09-22T01:14:55.903 INFO:teuthology.orchestra.run.smithi096.stdout:2375 read 0x832ed thru 0x8bba2 (0x88b6 bytes) 2024-09-22T01:14:55.903 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.903 INFO:teuthology.orchestra.run.smithi096.stdout:2378 punch from 0x6dc1bfb to 0x6dc22a4, (0x6a9 bytes) 2024-09-22T01:14:55.904 INFO:teuthology.orchestra.run.smithi096.stdout:2379 write 0xb8ba0a3 thru 0xb8c343a (0x9398 bytes) 2024-09-22T01:14:55.908 INFO:teuthology.orchestra.run.smithi096.stdout:2381 read 0xbbe81d5 thru 0xbbf3920 (0xb74c bytes) 2024-09-22T01:14:55.908 INFO:teuthology.orchestra.run.smithi096.stdout:2382 write 0x87633f thru 0x876afb (0x7bd bytes) 2024-09-22T01:14:55.912 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.912 INFO:teuthology.orchestra.run.smithi096.stdout:2384 read 0x78e88fc thru 0x78f0bda (0x82df bytes) 2024-09-22T01:14:55.928 INFO:teuthology.orchestra.run.smithi096.stdout:2386 trunc from 0xe14edb1 to 0x1ccc1ed 2024-09-22T01:14:55.947 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.956 INFO:teuthology.orchestra.run.smithi096.stdout:2389 write 0x352f609 thru 0x3534f0e (0x5906 bytes) 2024-09-22T01:14:55.964 INFO:teuthology.orchestra.run.smithi096.stdout:2390 read 0x22dee2b thru 0x22ebb67 (0xcd3d bytes) 2024-09-22T01:14:55.975 INFO:teuthology.orchestra.run.smithi096.stdout:2392 write 0x6537909 thru 0x654474c (0xce44 bytes) 2024-09-22T01:14:55.982 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:55.983 INFO:teuthology.orchestra.run.smithi096.stdout:2395 trunc from 0x654474d to 0x592fd6f 2024-09-22T01:14:55.986 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:55.982+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:55.990 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:55.990 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2397 punch from 0x3c31e06 to 0x3c3c3cb, (0xa5c5 bytes) 2024-09-22T01:14:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2398 read 0x29f4828 thru 0x2a0093f (0xc118 bytes) 2024-09-22T01:14:55.999 INFO:teuthology.orchestra.run.smithi096.stdout:2399 punch from 0x501a402 to 0x5028421, (0xe01f bytes) 2024-09-22T01:14:56.007 INFO:teuthology.orchestra.run.smithi096.stdout:2401 trunc from 0x592fd6f to 0xb863963 2024-09-22T01:14:56.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:14:56.015 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.016 INFO:teuthology.orchestra.run.smithi096.stdout:2405 write 0x4fa86e5 thru 0x4fb3751 (0xb06d bytes) 2024-09-22T01:14:56.021 INFO:teuthology.orchestra.run.smithi096.stdout:2406 writesame 0x7e62fb thru 0x7f343e data_size 0x4(0xd144 bytes) 2024-09-22T01:14:56.036 INFO:teuthology.orchestra.run.smithi096.stdout:2407 write 0x1a28987 thru 0x1a2fa5a (0x70d4 bytes) 2024-09-22T01:14:56.042 INFO:teuthology.orchestra.run.smithi096.stdout:2408 writesame 0x1b9089a thru 0x1b9a84d data_size 0x6(0x9fb4 bytes) 2024-09-22T01:14:56.050 INFO:teuthology.orchestra.run.smithi096.stdout:2410 writesame 0x36fe30b thru 0x36fef65 data_size 0x1(0xc5b bytes) 2024-09-22T01:14:56.057 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.057 INFO:teuthology.orchestra.run.smithi096.stdout:2414 punch from 0x7b22483 to 0x7b29c4f, (0x77cc bytes) 2024-09-22T01:14:56.067 INFO:teuthology.orchestra.run.smithi096.stdout:2415 trunc from 0xb863963 to 0x38ba2b2 2024-09-22T01:14:56.087 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.087 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:56.082+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:56.095 INFO:teuthology.orchestra.run.smithi096.stdout:2417 writesame 0x3d14b5b thru 0x3d1b0cc data_size 0x62(0x6572 bytes) 2024-09-22T01:14:56.106 INFO:teuthology.orchestra.run.smithi096.stdout:2418 writesame 0x443ffbe thru 0x444f275 data_size 0x24(0xf2b8 bytes) 2024-09-22T01:14:56.106 INFO:teuthology.orchestra.run.smithi096.stdout:2420 write 0x1720c3 thru 0x17bf9e (0x9edc bytes) 2024-09-22T01:14:56.110 INFO:teuthology.orchestra.run.smithi096.stdout:2422 read 0x3760602 thru 0x37631d7 (0x2bd6 bytes) 2024-09-22T01:14:56.110 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.111 INFO:teuthology.orchestra.run.smithi096.stdout:2425 punch from 0x3c1ad06 to 0x3c1f30d, (0x4607 bytes) 2024-09-22T01:14:56.111 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:2430 write 0x53b829e thru 0x53bb1ec (0x2f4f bytes) 2024-09-22T01:14:56.137 INFO:teuthology.orchestra.run.smithi096.stdout:2431 trunc from 0x53bb1ed to 0x8f3dd9d 2024-09-22T01:14:56.140 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.140 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.150 INFO:teuthology.orchestra.run.smithi096.stdout:2434 trunc from 0x8f3dd9d to 0x737038d 2024-09-22T01:14:56.156 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:14:56.161 INFO:teuthology.orchestra.run.smithi096.stdout:2436 read 0x3fdbae thru 0x403fac (0x63ff bytes) 2024-09-22T01:14:56.166 INFO:teuthology.orchestra.run.smithi096.stdout:2437 trunc from 0x737038d to 0x9915e58 2024-09-22T01:14:56.173 INFO:teuthology.orchestra.run.smithi096.stdout:2439 punch from 0x72e33d6 to 0x72f29d1, (0xf5fb bytes) 2024-09-22T01:14:56.173 INFO:teuthology.orchestra.run.smithi096.stdout:2440 write 0x1fb307c thru 0x1fb5900 (0x2885 bytes) 2024-09-22T01:14:56.180 INFO:teuthology.orchestra.run.smithi096.stdout:2441 trunc from 0x9915e58 to 0xabbe328 2024-09-22T01:14:56.186 INFO:teuthology.orchestra.run.smithi096.stdout:2443 read 0x4cf1d00 thru 0x4d0132a (0xf62b bytes) 2024-09-22T01:14:56.186 INFO:teuthology.orchestra.run.smithi096.stdout:2445 punch from 0x3ccec09 to 0x3cd9a1e, (0xae15 bytes) 2024-09-22T01:14:56.187 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:56.186+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:56.189 INFO:teuthology.orchestra.run.smithi096.stdout:2446 trunc from 0xabbe328 to 0x8c08817 2024-09-22T01:14:56.196 INFO:teuthology.orchestra.run.smithi096.stdout:2447 read 0x70fbe4b thru 0x7102ce2 (0x6e98 bytes) 2024-09-22T01:14:56.196 INFO:teuthology.orchestra.run.smithi096.stdout:2448 writesame 0x684897f thru 0x684f724 data_size 0x23(0x6da6 bytes) 2024-09-22T01:14:56.201 INFO:teuthology.orchestra.run.smithi096.stdout:2449 writesame 0x3f76fc4 thru 0x3f79072 data_size 0x1(0x20af bytes) 2024-09-22T01:14:56.209 INFO:teuthology.orchestra.run.smithi096.stdout:2450 writesame 0x55b8ac4 thru 0x55c4b1d data_size 0x6(0xc05a bytes) 2024-09-22T01:14:56.220 INFO:teuthology.orchestra.run.smithi096.stdout:2452 read 0x3567105 thru 0x35696a5 (0x25a1 bytes) 2024-09-22T01:14:56.220 INFO:teuthology.orchestra.run.smithi096.stdout:2453 write 0x661728e thru 0x662654e (0xf2c1 bytes) 2024-09-22T01:14:56.227 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.236 INFO:teuthology.orchestra.run.smithi096.stdout:2455 trunc from 0x8c08817 to 0xfe13ac 2024-09-22T01:14:56.255 INFO:teuthology.orchestra.run.smithi096.stdout:2457 punch from 0x1a10fe to 0x1a2c89, (0x1b8b bytes) 2024-09-22T01:14:56.279 INFO:teuthology.orchestra.run.smithi096.stdout:2458 write 0xc3e1ff7 thru 0xc3ee2d9 (0xc2e3 bytes) 2024-09-22T01:14:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:2459 read 0xaa12132 thru 0xaa15191 (0x3060 bytes) 2024-09-22T01:14:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:2461 read 0x9a4c9b6 thru 0x9a58d6c (0xc3b7 bytes) 2024-09-22T01:14:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:2462 read 0x89dfc4d thru 0x89e0eed (0x12a1 bytes) 2024-09-22T01:14:56.289 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:56.286+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:56.296 INFO:teuthology.orchestra.run.smithi096.stdout:2463 trunc from 0xc3ee2da to 0x4113cdf 2024-09-22T01:14:56.300 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.301 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:14:56.318 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.319 INFO:teuthology.orchestra.run.smithi096.stdout:2467 read 0x122f039 thru 0x122f898 (0x860 bytes) 2024-09-22T01:14:56.334 INFO:teuthology.orchestra.run.smithi096.stdout:2468 write 0x690cc8c thru 0x6912eaf (0x6224 bytes) 2024-09-22T01:14:56.343 INFO:teuthology.orchestra.run.smithi096.stdout:2469 writesame 0x704cb7 thru 0x70b489 data_size 0x1(0x67d3 bytes) 2024-09-22T01:14:56.383 INFO:teuthology.orchestra.run.smithi096.stdout:2470 trunc from 0x6912eb0 to 0x12974a 2024-09-22T01:14:56.390 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:56.387+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:56.415 INFO:teuthology.orchestra.run.smithi096.stdout:2471 punch from 0xcd4e1 to 0xd9a84, (0xc5a3 bytes) 2024-09-22T01:14:56.427 INFO:teuthology.orchestra.run.smithi096.stdout:2474 writesame 0x19e265d thru 0x19edbd0 data_size 0x9e(0xb574 bytes) 2024-09-22T01:14:56.433 INFO:teuthology.orchestra.run.smithi096.stdout:2475 trunc from 0x19edbd1 to 0x4e076ca 2024-09-22T01:14:56.442 INFO:teuthology.orchestra.run.smithi096.stdout:2476 read 0x4841de9 thru 0x48493a0 (0x75b8 bytes) 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:2478 read 0x2c732db thru 0x2c7a123 (0x6e49 bytes) 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:2479 read 0x11a09db thru 0x11a867a (0x7ca0 bytes) 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:2480 punch from 0x3f36d76 to 0x3f40bf2, (0x9e7c bytes) 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:2481 punch from 0x352c8cc to 0x35357ab, (0x8edf bytes) 2024-09-22T01:14:56.443 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:2484 writesame 0x8e4bde9 thru 0x8e52780 data_size 0x6d(0x6998 bytes) 2024-09-22T01:14:56.460 INFO:teuthology.orchestra.run.smithi096.stdout:2487 read 0x4f9768e thru 0x4f99a3d (0x23b0 bytes) 2024-09-22T01:14:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.461 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.474 INFO:teuthology.orchestra.run.smithi096.stdout:2488 write 0xd5f5d48 thru 0xd5f6288 (0x541 bytes) 2024-09-22T01:14:56.479 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:14:56.482 INFO:teuthology.orchestra.run.smithi096.stdout:2490 write 0x7b7880e thru 0x7b7ed94 (0x6587 bytes) 2024-09-22T01:14:56.486 INFO:teuthology.orchestra.run.smithi096.stdout:2491 writesame 0x5aad505 thru 0x5ab2c2a data_size 0xa(0x5726 bytes) 2024-09-22T01:14:56.490 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:56.486+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:56.496 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.496 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.496 INFO:teuthology.orchestra.run.smithi096.stdout:2494 read 0x3060774 thru 0x3065807 (0x5094 bytes) 2024-09-22T01:14:56.496 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.513 INFO:teuthology.orchestra.run.smithi096.stdout:2496 trunc from 0xd5f6289 to 0x174094 2024-09-22T01:14:56.539 INFO:teuthology.orchestra.run.smithi096.stdout:2497 read 0x13642b thru 0x13cca1 (0x6877 bytes) 2024-09-22T01:14:56.565 INFO:teuthology.orchestra.run.smithi096.stdout:2499 writesame 0xcffde62 thru 0xd00b08f data_size 0x1(0xd22e bytes) 2024-09-22T01:14:56.565 INFO:teuthology.orchestra.run.smithi096.stdout:2500 write 0x73f3eee thru 0x73f746f (0x3582 bytes) 2024-09-22T01:14:56.570 INFO:teuthology.orchestra.run.smithi096.stdout:2501 write 0x280f34a thru 0x28131fa (0x3eb1 bytes) 2024-09-22T01:14:56.586 INFO:teuthology.orchestra.run.smithi096.stdout:2502 trunc from 0xd00b090 to 0x7a685ef 2024-09-22T01:14:56.590 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:56.586+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:56.602 INFO:teuthology.orchestra.run.smithi096.stdout:2504 trunc from 0x7a685ef to 0x2ac41a2 2024-09-22T01:14:56.618 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.618 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.640 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:14:56.642 INFO:teuthology.orchestra.run.smithi096.stdout:2506 write 0xb2fa70c thru 0xb2fbf1e (0x1813 bytes) 2024-09-22T01:14:56.647 INFO:teuthology.orchestra.run.smithi096.stdout:2507 read 0x1f35cde thru 0x1f392e4 (0x3607 bytes) 2024-09-22T01:14:56.657 INFO:teuthology.orchestra.run.smithi096.stdout:2509 write 0xc99f7d2 thru 0xc9a87c0 (0x8fef bytes) 2024-09-22T01:14:56.663 INFO:teuthology.orchestra.run.smithi096.stdout:2510 read 0xae0aee8 thru 0xae14f33 (0xa04c bytes) 2024-09-22T01:14:56.664 INFO:teuthology.orchestra.run.smithi096.stdout:2512 read 0xba1722f thru 0xba1dc07 (0x69d9 bytes) 2024-09-22T01:14:56.664 INFO:teuthology.orchestra.run.smithi096.stdout:2513 write 0xbf13206 thru 0xbf19c19 (0x6a14 bytes) 2024-09-22T01:14:56.669 INFO:teuthology.orchestra.run.smithi096.stdout:2514 read 0xb75c501 thru 0xb75c95d (0x45d bytes) 2024-09-22T01:14:56.670 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.670 INFO:teuthology.orchestra.run.smithi096.stdout:2516 write 0xad56487 thru 0xad5b9b5 (0x552f bytes) 2024-09-22T01:14:56.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:14:56.674 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.674 INFO:teuthology.orchestra.run.smithi096.stdout:2523 punch from 0x7759c13 to 0x775b959, (0x1d46 bytes) 2024-09-22T01:14:56.674 INFO:teuthology.orchestra.run.smithi096.stdout:2524 read 0x9355134 thru 0x9356760 (0x162d bytes) 2024-09-22T01:14:56.674 INFO:teuthology.orchestra.run.smithi096.stdout:2525 read 0x342e31f thru 0x3436200 (0x7ee2 bytes) 2024-09-22T01:14:56.675 INFO:teuthology.orchestra.run.smithi096.stdout:2526 punch from 0x4404c59 to 0x441005e, (0xb405 bytes) 2024-09-22T01:14:56.675 INFO:teuthology.orchestra.run.smithi096.stdout:2527 read 0x8d31fb6 thru 0x8d31fc2 (0xd bytes) 2024-09-22T01:14:56.675 INFO:teuthology.orchestra.run.smithi096.stdout:2528 write 0x46548c8 thru 0x465a671 (0x5daa bytes) 2024-09-22T01:14:56.678 INFO:teuthology.orchestra.run.smithi096.stdout:2529 writesame 0x520effa thru 0x521a911 data_size 0x4(0xb918 bytes) 2024-09-22T01:14:56.691 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:56.686+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:56.696 INFO:teuthology.orchestra.run.smithi096.stdout:2531 read 0x5a35a5c thru 0x5a432f5 (0xd89a bytes) 2024-09-22T01:14:56.697 INFO:teuthology.orchestra.run.smithi096.stdout:2532 write 0xb49438b thru 0xb49a160 (0x5dd6 bytes) 2024-09-22T01:14:56.703 INFO:teuthology.orchestra.run.smithi096.stdout:2533 punch from 0x7080b79 to 0x7088fc1, (0x8448 bytes) 2024-09-22T01:14:56.703 INFO:teuthology.orchestra.run.smithi096.stdout:2535 write 0x13b59 thru 0x1808c (0x4534 bytes) 2024-09-22T01:14:56.705 INFO:teuthology.orchestra.run.smithi096.stdout:2539 write 0x57f612c thru 0x57f9bf7 (0x3acc bytes) 2024-09-22T01:14:56.710 INFO:teuthology.orchestra.run.smithi096.stdout:2540 writesame 0x53f0aef thru 0x53f445c data_size 0x1(0x396e bytes) 2024-09-22T01:14:56.728 INFO:teuthology.orchestra.run.smithi096.stdout:2542 read 0x211a780 thru 0x211d5ae (0x2e2f bytes) 2024-09-22T01:14:56.728 INFO:teuthology.orchestra.run.smithi096.stdout:2543 punch from 0x6e4556e to 0x6e4900b, (0x3a9d bytes) 2024-09-22T01:14:56.728 INFO:teuthology.orchestra.run.smithi096.stdout:2544 punch from 0xbda4e48 to 0xbda8e88, (0x4040 bytes) 2024-09-22T01:14:56.728 INFO:teuthology.orchestra.run.smithi096.stdout:2545 read 0x8cc148 thru 0x8d4f20 (0x8dd9 bytes) 2024-09-22T01:14:56.728 INFO:teuthology.orchestra.run.smithi096.stdout:2547 write 0x367d016 thru 0x368b53c (0xe527 bytes) 2024-09-22T01:14:56.739 INFO:teuthology.orchestra.run.smithi096.stdout:2550 punch from 0x8c9d41d to 0x8ca8865, (0xb448 bytes) 2024-09-22T01:14:56.739 INFO:teuthology.orchestra.run.smithi096.stdout:2551 write 0x2b12303 thru 0x2b160c8 (0x3dc6 bytes) 2024-09-22T01:14:56.743 INFO:teuthology.orchestra.run.smithi096.stdout:2552 read 0x46c0510 thru 0x46c06b1 (0x1a2 bytes) 2024-09-22T01:14:56.744 INFO:teuthology.orchestra.run.smithi096.stdout:2553 write 0xa1b5902 thru 0xa1b72ee (0x19ed bytes) 2024-09-22T01:14:56.755 INFO:teuthology.orchestra.run.smithi096.stdout:2554 trunc from 0xc9a87c1 to 0x7f97a04 2024-09-22T01:14:56.775 INFO:teuthology.orchestra.run.smithi096.stdout:2556 trunc from 0x7f97a04 to 0x7fb70f8 2024-09-22T01:14:56.777 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.777 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.793 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:56.790+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:56.793 INFO:teuthology.orchestra.run.smithi096.stdout:2557 trunc from 0x7fb70f8 to 0x19300ef 2024-09-22T01:14:56.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:14:56.829 INFO:teuthology.orchestra.run.smithi096.stdout:2558 write 0x9d615b1 thru 0x9d6dafa (0xc54a bytes) 2024-09-22T01:14:56.833 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:56.833 INFO:teuthology.orchestra.run.smithi096.stdout:2560 write 0x27de534 thru 0x27eddc6 (0xf893 bytes) 2024-09-22T01:14:56.837 INFO:teuthology.orchestra.run.smithi096.stdout:2561 read 0x49db17d thru 0x49eac59 (0xfadd bytes) 2024-09-22T01:14:56.837 INFO:teuthology.orchestra.run.smithi096.stdout:2562 read 0x58e918 thru 0x595e49 (0x7532 bytes) 2024-09-22T01:14:56.838 INFO:teuthology.orchestra.run.smithi096.stdout:2564 read 0x39af962 thru 0x39b3d6f (0x440e bytes) 2024-09-22T01:14:56.839 INFO:teuthology.orchestra.run.smithi096.stdout:2565 punch from 0x36f7f04 to 0x3705c4c, (0xdd48 bytes) 2024-09-22T01:14:56.849 INFO:teuthology.orchestra.run.smithi096.stdout:2570 trunc from 0x9d6dafb to 0x3ddcffe 2024-09-22T01:14:56.871 INFO:teuthology.orchestra.run.smithi096.stdout:2572 writesame 0x55d2944 thru 0x55d321a data_size 0x1(0x8d7 bytes) 2024-09-22T01:14:56.871 INFO:teuthology.orchestra.run.smithi096.stdout:2573 read 0xa0d265 thru 0xa11f27 (0x4cc3 bytes) 2024-09-22T01:14:56.876 INFO:teuthology.orchestra.run.smithi096.stdout:2574 trunc from 0x55d321b to 0x85ea162 2024-09-22T01:14:56.883 INFO:teuthology.orchestra.run.smithi096.stdout:2576 trunc from 0x85ea162 to 0x7f43959 2024-09-22T01:14:56.891 INFO:teuthology.orchestra.run.smithi096.stdout:2577 read 0x4a78bbf thru 0x4a7b3e8 (0x282a bytes) 2024-09-22T01:14:56.891 INFO:teuthology.orchestra.run.smithi096.stdout:2580 read 0x4a60733 thru 0x4a696b9 (0x8f87 bytes) 2024-09-22T01:14:56.894 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:56.890+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:56.910 INFO:teuthology.orchestra.run.smithi096.stdout:2582 writesame 0xe60bebd thru 0xe619959 data_size 0x29(0xda9d bytes) 2024-09-22T01:14:56.910 INFO:teuthology.orchestra.run.smithi096.stdout:2583 write 0x3ae8890 thru 0x3af6948 (0xe0b9 bytes) 2024-09-22T01:14:56.921 INFO:teuthology.orchestra.run.smithi096.stdout:2584 write 0x421790 thru 0x42e567 (0xcdd8 bytes) 2024-09-22T01:14:56.925 INFO:teuthology.orchestra.run.smithi096.stdout:2585 writesame 0xdacf2f3 thru 0xdad5817 data_size 0x1b(0x6525 bytes) 2024-09-22T01:14:56.932 INFO:teuthology.orchestra.run.smithi096.stdout:2588 read 0x648bcd7 thru 0x64957f8 (0x9b22 bytes) 2024-09-22T01:14:56.949 INFO:teuthology.orchestra.run.smithi096.stdout:2589 trunc from 0xe61995a to 0x3d93d00 2024-09-22T01:14:56.960 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:56.960 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:56.968 INFO:teuthology.orchestra.run.smithi096.stdout:2590 trunc from 0x3d93d00 to 0x37fccd0 2024-09-22T01:14:56.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:14:57.010 INFO:teuthology.orchestra.run.smithi096.stdout:2592 write 0xe1a2d0d thru 0xe1a9e6f (0x7163 bytes) 2024-09-22T01:14:57.010 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:57.006+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:57.014 INFO:teuthology.orchestra.run.smithi096.stdout:2593 writesame 0xa584123 thru 0xa5939c9 data_size 0x1d(0xf8a7 bytes) 2024-09-22T01:14:57.023 INFO:teuthology.orchestra.run.smithi096.stdout:2594 read 0x4165c5c thru 0x4174b2b (0xeed0 bytes) 2024-09-22T01:14:57.036 INFO:teuthology.orchestra.run.smithi096.stdout:2595 trunc from 0xe1a9e70 to 0x676bace 2024-09-22T01:14:57.050 INFO:teuthology.orchestra.run.smithi096.stdout:2596 writesame 0x2440b12 thru 0x244b018 data_size 0x1(0xa507 bytes) 2024-09-22T01:14:57.093 INFO:teuthology.orchestra.run.smithi096.stdout:2598 writesame 0xa18de67 thru 0xa18ec50 data_size 0xd(0xdea bytes) 2024-09-22T01:14:57.093 INFO:teuthology.orchestra.run.smithi096.stdout:2600 read 0x890a852 thru 0x8919d50 (0xf4ff bytes) 2024-09-22T01:14:57.098 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:57.094+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:57.104 INFO:teuthology.orchestra.run.smithi096.stdout:2602 write 0xc6b5c30 thru 0xc6bb57a (0x594b bytes) 2024-09-22T01:14:57.110 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.111 INFO:teuthology.orchestra.run.smithi096.stdout:2604 read 0x1e6b30f thru 0x1e78953 (0xd645 bytes) 2024-09-22T01:14:57.111 INFO:teuthology.orchestra.run.smithi096.stdout:2605 write 0x7d9a17c thru 0x7da889e (0xe723 bytes) 2024-09-22T01:14:57.119 INFO:teuthology.orchestra.run.smithi096.stdout:2606 write 0x2dd31c2 thru 0x2ddd435 (0xa274 bytes) 2024-09-22T01:14:57.124 INFO:teuthology.orchestra.run.smithi096.stdout:2607 writesame 0x9dcf268 thru 0x9dd1675 data_size 0x1a(0x240e bytes) 2024-09-22T01:14:57.132 INFO:teuthology.orchestra.run.smithi096.stdout:2609 write 0xb9732be thru 0xb9738cc (0x60f bytes) 2024-09-22T01:14:57.136 INFO:teuthology.orchestra.run.smithi096.stdout:2610 write 0x234ec01 thru 0x2351190 (0x2590 bytes) 2024-09-22T01:14:57.140 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:57.140 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:57.143 INFO:teuthology.orchestra.run.smithi096.stdout:2611 read 0x79b5ec2 thru 0x79c2d02 (0xce41 bytes) 2024-09-22T01:14:57.143 INFO:teuthology.orchestra.run.smithi096.stdout:2612 read 0xaad190e thru 0xaad3b17 (0x220a bytes) 2024-09-22T01:14:57.143 INFO:teuthology.orchestra.run.smithi096.stdout:2613 punch from 0x334e259 to 0x3352d18, (0x4abf bytes) 2024-09-22T01:14:57.143 INFO:teuthology.orchestra.run.smithi096.stdout:2615 writesame 0x6c1e434 thru 0x6c26546 data_size 0x1(0x8113 bytes) 2024-09-22T01:14:57.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:14:57.171 INFO:teuthology.orchestra.run.smithi096.stdout:2616 punch from 0xb838a93 to 0xb83a4ae, (0x1a1b bytes) 2024-09-22T01:14:57.183 INFO:teuthology.orchestra.run.smithi096.stdout:2620 write 0xec5e740 thru 0xec6b94f (0xd210 bytes) 2024-09-22T01:14:57.192 INFO:teuthology.orchestra.run.smithi096.stdout:2622 trunc from 0xec6b950 to 0xb455fea 2024-09-22T01:14:57.201 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:57.199+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:57.211 INFO:teuthology.orchestra.run.smithi096.stdout:2623 read 0xdcb270 thru 0xdd12db (0x606c bytes) 2024-09-22T01:14:57.212 INFO:teuthology.orchestra.run.smithi096.stdout:2624 read 0x3f0ec19 thru 0x3f1a53d (0xb925 bytes) 2024-09-22T01:14:57.216 INFO:teuthology.orchestra.run.smithi096.stdout:2625 trunc from 0xb455fea to 0xdab328c 2024-09-22T01:14:57.238 INFO:teuthology.orchestra.run.smithi096.stdout:2626 trunc from 0xdab328c to 0x7ca1a83 2024-09-22T01:14:57.256 INFO:teuthology.orchestra.run.smithi096.stdout:2627 punch from 0x5821ed8 to 0x582e290, (0xc3b8 bytes) 2024-09-22T01:14:57.256 INFO:teuthology.orchestra.run.smithi096.stdout:2628 read 0x3a1cf35 thru 0x3a296b7 (0xc783 bytes) 2024-09-22T01:14:57.280 INFO:teuthology.orchestra.run.smithi096.stdout:2629 write 0xdaf74b7 thru 0xdafabc1 (0x370b bytes) 2024-09-22T01:14:57.287 INFO:teuthology.orchestra.run.smithi096.stdout:2632 writesame 0x5bd8576 thru 0x5be28a3 data_size 0x1(0xa32e bytes) 2024-09-22T01:14:57.293 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:57.293 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:57.298 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:57.294+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:57.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:14:57.353 INFO:teuthology.orchestra.run.smithi096.stdout:2633 read 0x9995ebd thru 0x999b0a2 (0x51e6 bytes) 2024-09-22T01:14:57.355 INFO:teuthology.orchestra.run.smithi096.stdout:2634 write 0x808cb9b thru 0x8098177 (0xb5dd bytes) 2024-09-22T01:14:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:2635 punch from 0xb3c1690 to 0xb3c815a, (0x6aca bytes) 2024-09-22T01:14:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.365 INFO:teuthology.orchestra.run.smithi096.stdout:2638 read 0x689ea46 thru 0x68a824a (0x9805 bytes) 2024-09-22T01:14:57.366 INFO:teuthology.orchestra.run.smithi096.stdout:2639 write 0x88b9534 thru 0x88ba2cf (0xd9c bytes) 2024-09-22T01:14:57.373 INFO:teuthology.orchestra.run.smithi096.stdout:2640 writesame 0x250f7c5 thru 0x25164e6 data_size 0x3d(0x6d22 bytes) 2024-09-22T01:14:57.377 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.377 INFO:teuthology.orchestra.run.smithi096.stdout:2642 read 0x545ecc6 thru 0x54648e8 (0x5c23 bytes) 2024-09-22T01:14:57.392 INFO:teuthology.orchestra.run.smithi096.stdout:2643 write 0xe44dd63 thru 0xe45c4a3 (0xe741 bytes) 2024-09-22T01:14:57.399 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:57.395+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:57.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:14:57.402 INFO:teuthology.orchestra.run.smithi096.stdout:2644 writesame 0xab4290e thru 0xab4da4f data_size 0x9(0xb142 bytes) 2024-09-22T01:14:57.418 INFO:teuthology.orchestra.run.smithi096.stdout:2645 write 0xa61d814 thru 0xa628140 (0xa92d bytes) 2024-09-22T01:14:57.426 INFO:teuthology.orchestra.run.smithi096.stdout:2646 write 0xd1411fc thru 0xd1506b5 (0xf4ba bytes) 2024-09-22T01:14:57.432 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.439 INFO:teuthology.orchestra.run.smithi096.stdout:2648 trunc from 0xe45c4a4 to 0xb4ae209 2024-09-22T01:14:57.456 INFO:teuthology.orchestra.run.smithi096.stdout:2649 writesame 0x4128d94 thru 0x412f61d data_size 0x1(0x688a bytes) 2024-09-22T01:14:57.475 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:14:57.476 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:14:57.490 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:14:57.502 INFO:teuthology.orchestra.run.smithi096.stdout:2650 write 0xe4f8f79 thru 0xe4fad43 (0x1dcb bytes) 2024-09-22T01:14:57.508 INFO:teuthology.orchestra.run.smithi096.stdout:2652 write 0xba3898d thru 0xba47e9e (0xf512 bytes) 2024-09-22T01:14:57.517 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.517 INFO:teuthology.orchestra.run.smithi096.stdout:2654 punch from 0x76e450e to 0x76e9c81, (0x5773 bytes) 2024-09-22T01:14:57.518 INFO:teuthology.orchestra.run.smithi096.stdout:2655 read 0x8a2d8e8 thru 0x8a3482d (0x6f46 bytes) 2024-09-22T01:14:57.518 INFO:teuthology.orchestra.run.smithi096.stdout:2657 punch from 0xa0c0cd9 to 0xa0c18ad, (0xbd4 bytes) 2024-09-22T01:14:57.518 INFO:teuthology.orchestra.run.smithi096.stdout:2659 read 0x7a786c thru 0x7b54fe (0xdc93 bytes) 2024-09-22T01:14:57.518 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:57.528 INFO:teuthology.orchestra.run.smithi096.stdout:2661 write 0xe9f4f8f thru 0xe9f95b4 (0x4626 bytes) 2024-09-22T01:14:57.533 INFO:teuthology.orchestra.run.smithi096.stdout:2662 write 0x3062856 thru 0x306661b (0x3dc6 bytes) 2024-09-22T01:14:57.537 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:57.535+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:57.539 INFO:teuthology.orchestra.run.smithi096.stdout:2663 read 0xabf701a thru 0xabfb812 (0x47f9 bytes) 2024-09-22T01:14:57.556 INFO:teuthology.orchestra.run.smithi096.stdout:2664 trunc from 0xe9f95b5 to 0x2977305 2024-09-22T01:14:57.574 INFO:teuthology.orchestra.run.smithi096.stdout:2665 clone 24 order 21 su 131072 sc 7 2024-09-22T01:14:57.578 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:57.579 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:57.595 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:57.596 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:57.600 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:14:57.598+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:14:57.611 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:57.611 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:57.628 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:57.628 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:57.648 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:14:57.648 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:14:57.667 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:57.667 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:57.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:57.667 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:57.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20616 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 1998848~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312571453, 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051106650000000003, 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20616 2024-09-22T01:14:57.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352409+0000", 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352411+0000", 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.353e-06 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352416+0000", 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352438+0000", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1338000000000001e-05 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352455+0000", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7061000000000001e-05 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403119+0000", 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.050663779999999999 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403199+0000", 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0049999999999994e-05 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403516+0000", 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00031730499999999999 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20617 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2326528~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312505287, 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.05156525, 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.673 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20617 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352475+0000", 2024-09-22T01:14:57.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352477+0000", 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.316e-06 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352483+0000", 2024-09-22T01:14:57.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4729999999999997e-06 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352502+0000", 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8821000000000002e-05 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352516+0000", 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3685e-05 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403605+0000", 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051089330000000002 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.403698+0000", 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3304e-05 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404041+0000", 2024-09-22T01:14:57.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034232099999999998 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20618 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 2686976~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.31244525, 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051903137000000002, 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20618 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352535+0000", 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352536+0000", 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9999999999999996e-07 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352541+0000", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2050000000000001e-06 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352553+0000", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1022999999999999e-05 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352560+0000", 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9719999999999992e-06 2024-09-22T01:14:57.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404089+0000", 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051528662000000003 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404150+0000", 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0965999999999999e-05 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404439+0000", 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000288509 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312410188000001, 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:14:57.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:14:57.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312379918000001, 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312350344999999, 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.693 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:14:57.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:14:57.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312296254, 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:14:57.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312240410000001, 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.704 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:14:57.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.31218252, 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.712 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312120787000001, 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:14:57.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:14:57.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.312025324, 2024-09-22T01:14:57.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:57.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:14:57.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311963501000001, 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:14:57.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:57.726 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311872360999999, 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:14:57.730 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311810455, 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311749968000001, 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:14:57.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.740 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311692497999999, 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:14:57.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:14:57.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311635217999999, 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.745 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:14:57.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:14:57.749 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.311575693999998, 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.750 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:14:57.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.754 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.31152281, 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.755 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:14:57.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:14:57.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.302410248000001, 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:57.759 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:14:57.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:57.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:14:57.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:14:57.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:57.765 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:57.766 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:14:57.766 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:14:57.767 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:14:57.768 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:14:57.763+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:14:57.768 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:14:57.768 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:14:57.769 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:14:57.769 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:14:57.770 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:14:57.868 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:57.866+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:57.945 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:57.945 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:57.945 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:57.946 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:57.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:14:57.968 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:57.966+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:58.054 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-22T01:14:58.073 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:58.070+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:58.086 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:14:58.086 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:14:58.086 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-22T01:14:58.086 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:14:58.086 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:14:58.086 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-22T01:14:58.096 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.096 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:58.096 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:58.097 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:58.097 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:58.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:14:58.205 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:58.203+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:58.214 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.214 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:14:58.215 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:58.271 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:58.265+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:58.309 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.310 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81660 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 897024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.12916603600000001, 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063684256999999994, 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81660 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6100000000000002e-07 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179640+0000", 2024-09-22T01:14:58.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5629999999999999e-06 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179646+0000", 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8449999999999999e-06 2024-09-22T01:14:58.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179649+0000", 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7669999999999999e-06 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243122+0000", 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063473048000000004 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243214+0000", 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2478000000000005e-05 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243322+0000", 2024-09-22T01:14:58.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010829500000000001 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.320 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81662 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 905216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.129151405, 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063961117999999997, 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.321 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81662 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.22e-07 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179654+0000", 2024-09-22T01:14:58.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3340000000000001e-06 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179658+0000", 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4340000000000001e-06 2024-09-22T01:14:58.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179660+0000", 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5959999999999998e-06 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243370+0000", 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063710125000000006 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243487+0000", 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116728 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243614+0000", 2024-09-22T01:14:58.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012667900000000001 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.330 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81664 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 913408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.12914026000000001, 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064198330999999997, 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81664 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179665+0000", 2024-09-22T01:14:58.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179669+0000", 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2909999999999999e-06 2024-09-22T01:14:58.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179671+0000", 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5229999999999998e-06 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243664+0000", 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0639927 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243753+0000", 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9555999999999994e-05 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243862+0000", 2024-09-22T01:14:58.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010883 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.340 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81791 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 781312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119673531, 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063538149000000002, 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81791 2024-09-22T01:14:58.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4900000000000001e-07 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189133+0000", 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.148e-06 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189139+0000", 2024-09-22T01:14:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6060000000000002e-06 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189145+0000", 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6379999999999998e-06 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252453+0000", 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063307422000000002 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252558+0000", 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010567499999999999 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252669+0000", 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011031099999999999 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81792 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 789504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119652498, 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063793329999999995, 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81792 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.352 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5699999999999999e-07 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189153+0000", 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.486e-06 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189164+0000", 2024-09-22T01:14:58.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1012000000000001e-05 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189170+0000", 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.378e-06 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252719+0000", 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063549117000000002 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252829+0000", 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109796 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252945+0000", 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116284 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81793 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119629111, 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064022895999999996, 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81793 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.362 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189182+0000", 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5130000000000002e-06 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189186+0000", 2024-09-22T01:14:58.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1919999999999998e-06 2024-09-22T01:14:58.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189189+0000", 2024-09-22T01:14:58.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6520000000000002e-06 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252995+0000", 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063805929999999997 2024-09-22T01:14:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253089+0000", 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4630999999999996e-05 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253198+0000", 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108719 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81794 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.11961156000000001, 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064298414999999998, 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81794 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:14:58.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:14:58.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3099999999999999e-07 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189204+0000", 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0740000000000001e-05 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189208+0000", 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7729999999999999e-06 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189212+0000", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6559999999999998e-06 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253247+0000", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064035089000000003 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253364+0000", 2024-09-22T01:14:58.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011712700000000001 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253491+0000", 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126799 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81795 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 814080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119587533, 2024-09-22T01:14:58.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064526389000000003, 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81795 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1e-07 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189218+0000", 2024-09-22T01:14:58.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3430000000000001e-06 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189222+0000", 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4319999999999999e-06 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189224+0000", 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:14:58.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253544+0000", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064319935999999994 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253638+0000", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253743+0000", 2024-09-22T01:14:58.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010508100000000001 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81796 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 822272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119570517, 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064799000999999995, 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81796 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7700000000000001e-07 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189236+0000", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6700000000000001e-06 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189245+0000", 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9480000000000003e-06 2024-09-22T01:14:58.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189248+0000", 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8619999999999999e-06 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253793+0000", 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064544659000000004 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253907+0000", 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011349999999999999 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254033+0000", 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012608500000000001 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81797 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:14:58.385 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119518728, 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064994761999999998, 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81797 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189286+0000", 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189288+0000", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8080000000000001e-06 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189292+0000", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7589999999999997e-06 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189300+0000", 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4669999999999998e-06 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254085+0000", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064785331000000002 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254177+0000", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1456e-05 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254280+0000", 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010350599999999999 2024-09-22T01:14:58.389 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81798 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 838656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.11949915, 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065215094000000001, 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81798 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6199999999999999e-07 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189307+0000", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5e-06 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189310+0000", 2024-09-22T01:14:58.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189314+0000", 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4340000000000001e-06 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254328+0000", 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065014409999999995 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254453+0000", 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000124401 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254520+0000", 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7523999999999999e-05 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81799 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 846848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:14:58.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119486647, 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065462031000000004, 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81799 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:14:58.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189318+0000", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7099999999999998e-07 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189319+0000", 2024-09-22T01:14:58.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.322e-06 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189322+0000", 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.388e-06 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189325+0000", 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5299999999999999e-06 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254565+0000", 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065240252999999998 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254664+0000", 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8679999999999997e-05 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254780+0000", 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115587 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81800 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.1194757, 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065696737000000005, 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81800 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189329+0000", 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189330+0000", 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189333+0000", 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189336+0000", 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254815+0000", 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065479350000000006 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254911+0000", 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6380999999999998e-05 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255025+0000", 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113835 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81801 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119458523, 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065920638000000004, 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81801 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3500000000000002e-07 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189348+0000", 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6670000000000001e-06 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189360+0000", 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2582000000000001e-05 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189363+0000", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255063+0000", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065699750000000001 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255155+0000", 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2435000000000005e-05 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255266+0000", 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011106 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81802 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.11943648, 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066053509999999996, 2024-09-22T01:14:58.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81802 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4200000000000002e-07 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189369+0000", 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4130000000000001e-06 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189374+0000", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2479999999999998e-06 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189376+0000", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5560000000000002e-06 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255307+0000", 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065930585999999999 2024-09-22T01:14:58.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255360+0000", 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2852999999999997e-05 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255421+0000", 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1612000000000006e-05 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81803 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119423746, 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066282642000000003, 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81803 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189381+0000", 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189382+0000", 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.187e-06 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189385+0000", 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4999999999999999e-06 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189389+0000", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.304e-06 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255467+0000", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066078908000000006 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255560+0000", 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2607000000000003e-05 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255663+0000", 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010291899999999999 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81804 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119411797, 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066499758000000006, 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81804 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189393+0000", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189394+0000", 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:14:58.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189397+0000", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4470000000000002e-06 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189400+0000", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1030000000000002e-06 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255697+0000", 2024-09-22T01:14:58.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066297054999999994 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255787+0000", 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9646000000000002e-05 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255892+0000", 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105011 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81805 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119400214, 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066645572, 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81805 2024-09-22T01:14:58.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2000000000000001e-07 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189412+0000", 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9889999999999992e-06 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189416+0000", 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0890000000000002e-06 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189419+0000", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5720000000000001e-06 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255933+0000", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066514719999999999 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255988+0000", 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4400000000000001e-05 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256049+0000", 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1581999999999994e-05 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81806 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 904192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119379966, 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066870436000000005, 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81806 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:14:58.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.79e-07 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189426+0000", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189429+0000", 2024-09-22T01:14:58.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5710000000000002e-06 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189436+0000", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256094+0000", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066657328000000002 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256185+0000", 2024-09-22T01:14:58.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1278000000000003e-05 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256295+0000", 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109756 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81807 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 912384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.119364031, 2024-09-22T01:14:58.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.067081691999999998, 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81807 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3699999999999999e-07 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189442+0000", 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189447+0000", 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.916e-06 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189449+0000", 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5409999999999999e-06 2024-09-22T01:14:58.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256330+0000", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066880816999999995 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256418+0000", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7715999999999998e-05 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256522+0000", 2024-09-22T01:14:58.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010404900000000001 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.435 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:58.436 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:14:58.437 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:14:58.431+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:14:58.537 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:58.533+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:58.596 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:58.596 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:58.596 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:58.610 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:14:58.637 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:14:58.633+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:14:58.730 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:58.730 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:58.730 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:58.741 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:58.735+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:58.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:14:58.800 INFO:teuthology.orchestra.run.smithi096.stdout:2666 read 0x2251edc thru 0x22571e0 (0x5305 bytes) 2024-09-22T01:14:58.819 INFO:teuthology.orchestra.run.smithi096.stdout:2667 writesame 0x7160e82 thru 0x716c092 data_size 0xf(0xb211 bytes) 2024-09-22T01:14:58.832 INFO:teuthology.orchestra.run.smithi096.stdout:2668 write 0xc0ecdc3 thru 0xc0eea67 (0x1ca5 bytes) 2024-09-22T01:14:58.836 INFO:teuthology.orchestra.run.smithi096.stdout:2669 read 0x40d860c thru 0x40db601 (0x2ff6 bytes) 2024-09-22T01:14:58.836 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:58.839 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:58.837+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:58.845 INFO:teuthology.orchestra.run.smithi096.stdout:2672 writesame 0xde3a59f thru 0xde45c1e data_size 0x92(0xb680 bytes) 2024-09-22T01:14:58.853 INFO:teuthology.orchestra.run.smithi096.stdout:2673 write 0xe296641 thru 0xe29e219 (0x7bd9 bytes) 2024-09-22T01:14:58.864 INFO:teuthology.orchestra.run.smithi096.stdout:2675 trunc from 0xe29e21a to 0xa565a2d 2024-09-22T01:14:58.880 INFO:teuthology.orchestra.run.smithi096.stdout:2676 write 0x881ee11 thru 0x882d831 (0xea21 bytes) 2024-09-22T01:14:58.898 INFO:teuthology.orchestra.run.smithi096.stdout:2677 trunc from 0xa565a2d to 0x36860f 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.905 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591869715999998, 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:58.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:14:58.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.909 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591817495999997, 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:14:58.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:14:58.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:14:58.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:14:58.913 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591617847999999, 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.914 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591539144999999, 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591479151999998, 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591406659, 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:14:58.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591338346999997, 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591280423000001, 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:58.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:14:58.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:14:58.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.591224533999998, 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:14:58.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:14:58.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.704279756999998, 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:14:58.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:14:58.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:14:58.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:14:58.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.008912776999999, 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:14:58.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.616787766000002, 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.924933518, 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:14:58.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:14:58.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:14:58.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:14:58.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.924815298999999, 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:14:58.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.922288357999999, 2024-09-22T01:14:58.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:14:58.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:14:58.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.903842005000001, 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:14:58.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:14:58.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.744215111999999, 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:14:58.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:14:58.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:14:58.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:14:58.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.744161017, 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:14:58.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:14:58.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.744074721, 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:58.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:58.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:14:58.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:14:58.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:58.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.000 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.743445011999999, 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.006 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:59.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2680 read 0x28c02c thru 0x2926c3 (0x6698 bytes) 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2682 read 0x16c70b thru 0x16ef7e (0x2874 bytes) 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2684 write 0x18e61a7 thru 0x18f1321 (0xb17b bytes) 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2685 trunc from 0x18f1322 to 0xd3e9ada 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2687 trunc from 0xd3e9ada to 0x7347c40 2024-09-22T01:14:59.009 INFO:teuthology.orchestra.run.smithi096.stdout:2688 punch from 0x6a8fc4c to 0x6a9b17d, (0xb531 bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2689 punch from 0x71a8499 to 0x71adb8d, (0x56f4 bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2690 trunc from 0x7347c40 to 0x87ab45e 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2692 read 0x6f5fd1b thru 0x6f64a5e (0x4d44 bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2693 writesame 0x5a2a504 thru 0x5a2eda9 data_size 0x11(0x48a6 bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2696 read 0x331682f thru 0x331a70f (0x3ee1 bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2697 writesame 0x665459d thru 0x6663816 data_size 0x29(0xf27a bytes) 2024-09-22T01:14:59.010 INFO:teuthology.orchestra.run.smithi096.stdout:2698 writesame 0x4df1313 thru 0x4df4e2d data_size 0x1(0x3b1b bytes) 2024-09-22T01:14:59.010 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:59.001+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:59.028 INFO:teuthology.orchestra.run.smithi096.stdout:2701 read 0x5bd20c7 thru 0x5bd8338 (0x6272 bytes) 2024-09-22T01:14:59.045 INFO:teuthology.orchestra.run.smithi096.stdout:2703 writesame 0xb487b82 thru 0xb48d182 data_size 0x3(0x5601 bytes) 2024-09-22T01:14:59.045 INFO:teuthology.orchestra.run.smithi096.stdout:2704 read 0x3b5690c thru 0x3b64974 (0xe069 bytes) 2024-09-22T01:14:59.045 INFO:teuthology.orchestra.run.smithi096.stdout:2705 read 0x6a8045 thru 0x6ae135 (0x60f1 bytes) 2024-09-22T01:14:59.046 INFO:teuthology.orchestra.run.smithi096.stdout:2706 read 0x8a55395 thru 0x8a55edf (0xb4b bytes) 2024-09-22T01:14:59.046 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.046 INFO:teuthology.orchestra.run.smithi096.stdout:2708 read 0x5804c29 thru 0x580e566 (0x993e bytes) 2024-09-22T01:14:59.047 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.068 INFO:teuthology.orchestra.run.smithi096.stdout:2710 trunc from 0xb48d183 to 0x1e186a1 2024-09-22T01:14:59.108 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:14:59.103+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:14:59.129 INFO:teuthology.orchestra.run.smithi096.stdout:2711 write 0xdadee26 thru 0xdae9f0f (0xb0ea bytes) 2024-09-22T01:14:59.138 INFO:teuthology.orchestra.run.smithi096.stdout:2712 write 0xcc67e59 thru 0xcc6b431 (0x35d9 bytes) 2024-09-22T01:14:59.139 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:59.139 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:14:59.139 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:59.145 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.145 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.145 INFO:teuthology.orchestra.run.smithi096.stdout:2715 read 0x45a22b2 thru 0x45a7e58 (0x5ba7 bytes) 2024-09-22T01:14:59.145 INFO:teuthology.orchestra.run.smithi096.stdout:2716 write 0xc9ae036 thru 0xc9b0af7 (0x2ac2 bytes) 2024-09-22T01:14:59.153 INFO:teuthology.orchestra.run.smithi096.stdout:2718 read 0x3aae740 thru 0x3ab0ca5 (0x2566 bytes) 2024-09-22T01:14:59.153 INFO:teuthology.orchestra.run.smithi096.stdout:2719 writesame 0x9b0b112 thru 0x9b0b1cd data_size 0x2(0xbc bytes) 2024-09-22T01:14:59.155 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:14:59.189 INFO:teuthology.orchestra.run.smithi096.stdout:2720 trunc from 0xdae9f10 to 0x21e4154 2024-09-22T01:14:59.212 INFO:teuthology.orchestra.run.smithi096.stdout:2723 read 0x4b0559 thru 0x4b0f1b (0x9c3 bytes) 2024-09-22T01:14:59.213 INFO:teuthology.orchestra.run.smithi096.stdout:2724 punch from 0x234152 to 0x235342, (0x11f0 bytes) 2024-09-22T01:14:59.239 INFO:teuthology.orchestra.run.smithi096.stdout:2726 write 0x7d348aa thru 0x7d3b258 (0x69af bytes) 2024-09-22T01:14:59.241 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:14:59.239+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:14:59.261 INFO:teuthology.orchestra.run.smithi096.stdout:2727 trunc from 0x7d3b259 to 0x17ef334 2024-09-22T01:14:59.285 INFO:teuthology.orchestra.run.smithi096.stdout:2728 write 0x504191 thru 0x506899 (0x2709 bytes) 2024-09-22T01:14:59.289 INFO:teuthology.orchestra.run.smithi096.stdout:2730 write 0xf132cb thru 0xf1a3f6 (0x712c bytes) 2024-09-22T01:14:59.296 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.309 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:59.305+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:59.317 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:14:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:14:59.317 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:14:59.317 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:59.332 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:14:59.341 INFO:teuthology.orchestra.run.smithi096.stdout:2732 writesame 0xe894e1b thru 0xe899c39 data_size 0x1(0x4e1f bytes) 2024-09-22T01:14:59.341 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.342 INFO:teuthology.orchestra.run.smithi096.stdout:2735 write 0xd390ef6 thru 0xd39610d (0x5218 bytes) 2024-09-22T01:14:59.348 INFO:teuthology.orchestra.run.smithi096.stdout:2736 write 0x98296f thru 0x983ba4 (0x1236 bytes) 2024-09-22T01:14:59.352 INFO:teuthology.orchestra.run.smithi096.stdout:2737 read 0xbbc0978 thru 0xbbce88c (0xdf15 bytes) 2024-09-22T01:14:59.352 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.352 INFO:teuthology.orchestra.run.smithi096.stdout:2741 write 0xb3ef753 thru 0xb3f228d (0x2b3b bytes) 2024-09-22T01:14:59.359 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:2743 trunc from 0xe899c3a to 0xc764707 2024-09-22T01:14:59.387 INFO:teuthology.orchestra.run.smithi096.stdout:2744 read 0x2b39fae thru 0x2b435d4 (0x9627 bytes) 2024-09-22T01:14:59.400 INFO:teuthology.orchestra.run.smithi096.stdout:2746 write 0xd3f6ab5 thru 0xd400cd4 (0xa220 bytes) 2024-09-22T01:14:59.407 INFO:teuthology.orchestra.run.smithi096.stdout:2747 punch from 0x74acb53 to 0x74b9082, (0xc52f bytes) 2024-09-22T01:14:59.407 INFO:teuthology.orchestra.run.smithi096.stdout:2748 write 0x9e149d5 thru 0x9e1f4fe (0xab2a bytes) 2024-09-22T01:14:59.410 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:59.405+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:59.414 INFO:teuthology.orchestra.run.smithi096.stdout:2749 writesame 0x31570e9 thru 0x3159aa8 data_size 0x20(0x29c0 bytes) 2024-09-22T01:14:59.422 INFO:teuthology.orchestra.run.smithi096.stdout:2750 read 0xc3b8b55 thru 0xc3bd4c9 (0x4975 bytes) 2024-09-22T01:14:59.422 INFO:teuthology.orchestra.run.smithi096.stdout:2751 punch from 0x2585109 to 0x258e1f9, (0x90f0 bytes) 2024-09-22T01:14:59.422 INFO:teuthology.orchestra.run.smithi096.stdout:2753 read 0xa5113fe thru 0xa51281a (0x141d bytes) 2024-09-22T01:14:59.423 INFO:teuthology.orchestra.run.smithi096.stdout:2755 punch from 0x41caf94 to 0x41cccd4, (0x1d40 bytes) 2024-09-22T01:14:59.423 INFO:teuthology.orchestra.run.smithi096.stdout:2756 punch from 0x63d533c to 0x63e4bb7, (0xf87b bytes) 2024-09-22T01:14:59.423 INFO:teuthology.orchestra.run.smithi096.stdout:2759 punch from 0x4ac3170 to 0x4acd085, (0x9f15 bytes) 2024-09-22T01:14:59.456 INFO:teuthology.orchestra.run.smithi096.stdout:2760 trunc from 0xd400cd5 to 0x6a686b 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:59.505 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.277159898999997, 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:14:59.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.277085069999998, 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:14:59.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.516 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.277047046, 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:14:59.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:59.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:59.522 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.276997823999999, 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:59.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:14:59.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:14:59.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.276961477, 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:14:59.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.533 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.618088586999999, 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.534 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.615907840999999, 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:14:59.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:14:59.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:14:59.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.615835356000002, 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:14:59.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.615773538999999, 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.212605960000001, 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535924178999998, 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:14:59.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535868554, 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:14:59.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535765934, 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:14:59.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535665445999999, 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535604879000001, 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:14:59.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535547504, 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:14:59.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:14:59.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535478793999999, 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:14:59.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:14:59.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535416850000001, 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:14:59.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.535352883000002, 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:14:59.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:14:59.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 15.679257743000001, 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:14:59.596 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:14:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:14:59.600 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:14:59.602 INFO:teuthology.orchestra.run.smithi096.stdout:2761 write 0xd02b2b2 thru 0xd02b8ad (0x5fc bytes) 2024-09-22T01:14:59.602 INFO:teuthology.orchestra.run.smithi096.stdout:2762 trunc from 0xd02b8ae to 0x6acc399 2024-09-22T01:14:59.602 INFO:teuthology.orchestra.run.smithi096.stdout:2763 trunc from 0x6acc399 to 0xc31abd9 2024-09-22T01:14:59.602 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:59.597+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:59.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:14:59.609 INFO:teuthology.orchestra.run.smithi096.stdout:2764 write 0x2b5f632 thru 0x2b63a33 (0x4402 bytes) 2024-09-22T01:14:59.619 INFO:teuthology.orchestra.run.smithi096.stdout:2765 punch from 0x9e5be60 to 0x9e61815, (0x59b5 bytes) 2024-09-22T01:14:59.619 INFO:teuthology.orchestra.run.smithi096.stdout:2766 read 0x772640f thru 0x7734e7d (0xea6f bytes) 2024-09-22T01:14:59.619 INFO:teuthology.orchestra.run.smithi096.stdout:2767 writesame 0xb47c574 thru 0xb4868db data_size 0x7e(0xa368 bytes) 2024-09-22T01:14:59.629 INFO:teuthology.orchestra.run.smithi096.stdout:2770 trunc from 0xc31abd9 to 0xb1eb9be 2024-09-22T01:14:59.644 INFO:teuthology.orchestra.run.smithi096.stdout:2772 read 0x6cde9d9 thru 0x6ce7fc3 (0x95eb bytes) 2024-09-22T01:14:59.644 INFO:teuthology.orchestra.run.smithi096.stdout:2774 write 0x140c5a9 thru 0x1414cc9 (0x8721 bytes) 2024-09-22T01:14:59.650 INFO:teuthology.orchestra.run.smithi096.stdout:2775 read 0x1a1ab20 thru 0x1a219f2 (0x6ed3 bytes) 2024-09-22T01:14:59.650 INFO:teuthology.orchestra.run.smithi096.stdout:2776 writesame 0x163946f thru 0x16468ee data_size 0x80(0xd480 bytes) 2024-09-22T01:14:59.660 INFO:teuthology.orchestra.run.smithi096.stdout:2779 writesame 0x757a593 thru 0x7588692 data_size 0xc8(0xe100 bytes) 2024-09-22T01:14:59.669 INFO:teuthology.orchestra.run.smithi096.stdout:2781 read 0x3e07cbd thru 0x3e0ec76 (0x6fba bytes) 2024-09-22T01:14:59.669 INFO:teuthology.orchestra.run.smithi096.stdout:2782 punch from 0x7b842c6 to 0x7b9143d, (0xd177 bytes) 2024-09-22T01:14:59.669 INFO:teuthology.orchestra.run.smithi096.stdout:2783 read 0x96114e2 thru 0x9615e5a (0x4979 bytes) 2024-09-22T01:14:59.669 INFO:teuthology.orchestra.run.smithi096.stdout:2786 write 0x475f06f thru 0x476ea52 (0xf9e4 bytes) 2024-09-22T01:14:59.695 INFO:teuthology.orchestra.run.smithi096.stdout:2788 write 0xbea0de9 thru 0xbea1003 (0x21b bytes) 2024-09-22T01:14:59.700 INFO:teuthology.orchestra.run.smithi096.stdout:2789 write 0x3ff3d09 thru 0x3ffa08a (0x6382 bytes) 2024-09-22T01:14:59.702 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:14:59.697+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:14:59.705 INFO:teuthology.orchestra.run.smithi096.stdout:2790 read 0x2ac3653 thru 0x2ac7892 (0x4240 bytes) 2024-09-22T01:14:59.705 INFO:teuthology.orchestra.run.smithi096.stdout:2791 write 0x84ec362 thru 0x84ee204 (0x1ea3 bytes) 2024-09-22T01:14:59.710 INFO:teuthology.orchestra.run.smithi096.stdout:2792 writesame 0x266ad52 thru 0x26770b9 data_size 0x1a(0xc368 bytes) 2024-09-22T01:14:59.718 INFO:teuthology.orchestra.run.smithi096.stdout:2795 read 0x83bc887 thru 0x83bc967 (0xe1 bytes) 2024-09-22T01:14:59.718 INFO:teuthology.orchestra.run.smithi096.stdout:2796 write 0x8eba2d0 thru 0x8ec03bb (0x60ec bytes) 2024-09-22T01:14:59.728 INFO:teuthology.orchestra.run.smithi096.stdout:2797 write 0x854e8b6 thru 0x854f3ad (0xaf8 bytes) 2024-09-22T01:14:59.734 INFO:teuthology.orchestra.run.smithi096.stdout:2798 punch from 0x97c8f62 to 0x97d0b6a, (0x7c08 bytes) 2024-09-22T01:14:59.734 INFO:teuthology.orchestra.run.smithi096.stdout:2799 read 0x3b15437 thru 0x3b18bdb (0x37a5 bytes) 2024-09-22T01:14:59.734 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.741 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:59.742 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:59.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:14:59.742 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:59.748 INFO:teuthology.orchestra.run.smithi096.stdout:2802 writesame 0xdda44e4 thru 0xdda4948 data_size 0x9(0x465 bytes) 2024-09-22T01:14:59.748 INFO:teuthology.orchestra.run.smithi096.stdout:2804 punch from 0x53d6bf1 to 0x53dd331, (0x6740 bytes) 2024-09-22T01:14:59.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:14:59.760 INFO:teuthology.orchestra.run.smithi096.stdout:2805 write 0xe532ba7 thru 0xe534f1a (0x2374 bytes) 2024-09-22T01:14:59.766 INFO:teuthology.orchestra.run.smithi096.stdout:2807 read 0x82b16fb thru 0x82bc444 (0xad4a bytes) 2024-09-22T01:14:59.766 INFO:teuthology.orchestra.run.smithi096.stdout:2808 writesame 0x42ffbd3 thru 0x43028b9 data_size 0x37(0x2ce7 bytes) 2024-09-22T01:14:59.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:14:59.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:14:59.775 INFO:teuthology.orchestra.run.smithi096.stdout:2809 writesame 0x112bef5 thru 0x112ccc8 data_size 0xf(0xdd4 bytes) 2024-09-22T01:14:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.781 INFO:teuthology.orchestra.run.smithi096.stdout:2811 writesame 0xa9896d4 thru 0xa98a05f data_size 0x1a(0x98c bytes) 2024-09-22T01:14:59.788 INFO:teuthology.orchestra.run.smithi096.stdout:2812 writesame 0x35d2595 thru 0x35e1d09 data_size 0xd(0xf775 bytes) 2024-09-22T01:14:59.794 INFO:teuthology.orchestra.run.smithi096.stdout:2813 punch from 0x3b4bc95 to 0x3b5239d, (0x6708 bytes) 2024-09-22T01:14:59.794 INFO:teuthology.orchestra.run.smithi096.stdout:2815 write 0x41e38fe thru 0x41eda76 (0xa179 bytes) 2024-09-22T01:14:59.801 INFO:teuthology.orchestra.run.smithi096.stdout:2817 read 0x701c806 thru 0x7020413 (0x3c0e bytes) 2024-09-22T01:14:59.802 INFO:teuthology.orchestra.run.smithi096.stdout:2820 write 0xb9d7b31 thru 0xb9e0c5b (0x912b bytes) 2024-09-22T01:14:59.803 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:59.797+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:59.812 INFO:teuthology.orchestra.run.smithi096.stdout:2821 trunc from 0xe534f1b to 0xd132605 2024-09-22T01:14:59.845 INFO:teuthology.orchestra.run.smithi096.stdout:2822 trunc from 0xd132605 to 0x5412c8e 2024-09-22T01:14:59.869 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.869 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.870 INFO:teuthology.orchestra.run.smithi096.stdout:2827 write 0x258135b thru 0x258234f (0xff5 bytes) 2024-09-22T01:14:59.872 INFO:teuthology.orchestra.run.smithi096.stdout:2828 read 0x53f1390 thru 0x53f837a (0x6feb bytes) 2024-09-22T01:14:59.872 INFO:teuthology.orchestra.run.smithi096.stdout:2829 read 0x3f09760 thru 0x3f12d5b (0x95fc bytes) 2024-09-22T01:14:59.877 INFO:teuthology.orchestra.run.smithi096.stdout:2830 trunc from 0x5412c8e to 0x29c443c 2024-09-22T01:14:59.904 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:14:59.901+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:14:59.917 INFO:teuthology.orchestra.run.smithi096.stdout:2831 trunc from 0x29c443c to 0xcc7b4a8 2024-09-22T01:14:59.918 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:14:59.918 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:14:59.918 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:14:59.918 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:14:59.918 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:14:59.927 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:14:59.927 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:14:59.927 INFO:teuthology.orchestra.run.smithi096.stdout:2833 read 0x5b7ed0d thru 0x5b81402 (0x26f6 bytes) 2024-09-22T01:14:59.928 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.928 INFO:teuthology.orchestra.run.smithi096.stdout:2835 write 0x74e20fe thru 0x74f1cd3 (0xfbd6 bytes) 2024-09-22T01:14:59.934 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:14:59.938 INFO:teuthology.orchestra.run.smithi096.stdout:2836 trunc from 0xcc7b4a8 to 0xa529dae 2024-09-22T01:14:59.946 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:14:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:14:59.949 INFO:teuthology.orchestra.run.smithi096.stdout:2837 punch from 0x717a00f to 0x71865b3, (0xc5a4 bytes) 2024-09-22T01:14:59.949 INFO:teuthology.orchestra.run.smithi096.stdout:2838 write 0x1288eb3 thru 0x12896d3 (0x821 bytes) 2024-09-22T01:14:59.955 INFO:teuthology.orchestra.run.smithi096.stdout:2841 writesame 0x7239ccd thru 0x7244c67 data_size 0x87(0xaf9b bytes) 2024-09-22T01:14:59.964 INFO:teuthology.orchestra.run.smithi096.stdout:2843 trunc from 0xa529dae to 0x890e322 2024-09-22T01:14:59.967 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:14:59.967 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:14:59.975 INFO:teuthology.orchestra.run.smithi096.stdout:2844 writesame 0x61018b3 thru 0x6107024 data_size 0x4e(0x5772 bytes) 2024-09-22T01:14:59.982 INFO:teuthology.orchestra.run.smithi096.stdout:2845 writesame 0x803a962 thru 0x8043749 data_size 0x4c(0x8de8 bytes) 2024-09-22T01:14:59.988 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:14:59.988 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:14:59.988 INFO:teuthology.orchestra.run.smithi096.stdout:2848 write 0x4b1dd2 thru 0x4b2c2b (0xe5a bytes) 2024-09-22T01:14:59.990 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:14:59.990 INFO:teuthology.orchestra.run.smithi096.stdout:2850 punch from 0x6d80f38 to 0x6d88a6e, (0x7b36 bytes) 2024-09-22T01:14:59.990 INFO:teuthology.orchestra.run.smithi096.stdout:2851 read 0x2fc069f thru 0x2fc3529 (0x2e8b bytes) 2024-09-22T01:14:59.990 INFO:teuthology.orchestra.run.smithi096.stdout:2853 writesame 0x8621aef thru 0x862b6df data_size 0x7(0x9bf1 bytes) 2024-09-22T01:15:00.000 INFO:teuthology.orchestra.run.smithi096.stdout:2855 write 0x88839e5 thru 0x8886554 (0x2b70 bytes) 2024-09-22T01:15:00.005 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:00.001+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:00.006 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.006 INFO:teuthology.orchestra.run.smithi096.stdout:2857 write 0x6140d5f thru 0x6141098 (0x33a bytes) 2024-09-22T01:15:00.011 INFO:teuthology.orchestra.run.smithi096.stdout:2859 write 0x2a2ac9f thru 0x2a2f80f (0x4b71 bytes) 2024-09-22T01:15:00.016 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.016 INFO:teuthology.orchestra.run.smithi096.stdout:2861 read 0x3d18c3d thru 0x3d24760 (0xbb24 bytes) 2024-09-22T01:15:00.018 INFO:teuthology.orchestra.run.smithi096.stdout:2862 trunc from 0x890e322 to 0x70af888 2024-09-22T01:15:00.024 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:00.024 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:00.040 INFO:teuthology.orchestra.run.smithi096.stdout:2863 trunc from 0x70af888 to 0x6f8a25a 2024-09-22T01:15:00.060 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:00.060 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:00.069 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.069 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.078 INFO:teuthology.orchestra.run.smithi096.stdout:2867 writesame 0xce489c1 thru 0xce57dec data_size 0x6(0xf42c bytes) 2024-09-22T01:15:00.078 INFO:teuthology.orchestra.run.smithi096.stdout:2869 punch from 0x64082dc to 0x6414fd2, (0xccf6 bytes) 2024-09-22T01:15:00.078 INFO:teuthology.orchestra.run.smithi096.stdout:2870 read 0x936d205 thru 0x937d07e (0xfe7a bytes) 2024-09-22T01:15:00.078 INFO:teuthology.orchestra.run.smithi096.stdout:2873 write 0x7bb9501 thru 0x7bc83db (0xeedb bytes) 2024-09-22T01:15:00.084 INFO:teuthology.orchestra.run.smithi096.stdout:2875 read 0x3a9d956 thru 0x3a9e6b7 (0xd62 bytes) 2024-09-22T01:15:00.084 INFO:teuthology.orchestra.run.smithi096.stdout:2876 write 0x649acd2 thru 0x64a0ca5 (0x5fd4 bytes) 2024-09-22T01:15:00.085 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:00.092 INFO:teuthology.orchestra.run.smithi096.stdout:2877 read 0x4814b3c thru 0x48160d0 (0x1595 bytes) 2024-09-22T01:15:00.096 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:00.096 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:00.100 INFO:teuthology.orchestra.run.smithi096.stdout:2879 trunc from 0xce57ded to 0x79c51c8 2024-09-22T01:15:00.106 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:00.101+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:00.124 INFO:teuthology.orchestra.run.smithi096.stdout:2880 write 0xaf7977c thru 0xaf8894c (0xf1d1 bytes) 2024-09-22T01:15:00.129 INFO:teuthology.orchestra.run.smithi096.stdout:2881 write 0x75f33 thru 0x7855c (0x262a bytes) 2024-09-22T01:15:00.132 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:00.132 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:00.133 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.133 INFO:teuthology.orchestra.run.smithi096.stdout:2884 write 0x28daf1f thru 0x28e83b4 (0xd496 bytes) 2024-09-22T01:15:00.140 INFO:teuthology.orchestra.run.smithi096.stdout:2885 punch from 0xa33ece3 to 0xa34a5e3, (0xb900 bytes) 2024-09-22T01:15:00.140 INFO:teuthology.orchestra.run.smithi096.stdout:2886 read 0x5b0bcc4 thru 0x5b12298 (0x65d5 bytes) 2024-09-22T01:15:00.140 INFO:teuthology.orchestra.run.smithi096.stdout:2888 write 0x58cd0b3 thru 0x58ce93b (0x1889 bytes) 2024-09-22T01:15:00.145 INFO:teuthology.orchestra.run.smithi096.stdout:2889 write 0x1005e80 thru 0x100bd45 (0x5ec6 bytes) 2024-09-22T01:15:00.148 INFO:teuthology.orchestra.run.smithi096.stdout:2890 writesame 0x17af629 thru 0x17b49a5 data_size 0x1d(0x537d bytes) 2024-09-22T01:15:00.154 INFO:teuthology.orchestra.run.smithi096.stdout:2891 read 0x407f998 thru 0x408bd97 (0xc400 bytes) 2024-09-22T01:15:00.158 INFO:teuthology.orchestra.run.smithi096.stdout:2892 trunc from 0xaf8894d to 0xdf1deaf 2024-09-22T01:15:00.167 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.167 INFO:teuthology.orchestra.run.smithi096.stdout:2895 read 0x3971d9e thru 0x397e90f (0xcb72 bytes) 2024-09-22T01:15:00.167 INFO:teuthology.orchestra.run.smithi096.stdout:2896 read 0x55073aa thru 0x55100d9 (0x8d30 bytes) 2024-09-22T01:15:00.177 INFO:teuthology.orchestra.run.smithi096.stdout:2900 writesame 0xeaefc6b thru 0xeafe1b2 data_size 0x5c(0xe548 bytes) 2024-09-22T01:15:00.177 INFO:teuthology.orchestra.run.smithi096.stdout:2901 write 0xe841ca7 thru 0xe84a131 (0x848b bytes) 2024-09-22T01:15:00.199 INFO:teuthology.orchestra.run.smithi096.stdout:2902 trunc from 0xeafe1b3 to 0x2065f83 2024-09-22T01:15:00.207 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:00.201+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:00.222 INFO:teuthology.orchestra.run.smithi096.stdout:2903 read 0x1a247b2 thru 0x1a311f7 (0xca46 bytes) 2024-09-22T01:15:00.222 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.222 INFO:teuthology.orchestra.run.smithi096.stdout:2905 read 0x1ad6717 thru 0x1add42d (0x6d17 bytes) 2024-09-22T01:15:00.223 INFO:teuthology.orchestra.run.smithi096.stdout:2906 punch from 0x5b6db7 to 0x5ba2f8, (0x3541 bytes) 2024-09-22T01:15:00.223 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.228 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.229 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.250 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:00.250 INFO:teuthology.orchestra.run.smithi096.stdout:2908 write 0xd1d18f2 thru 0xd1dadcb (0x94da bytes) 2024-09-22T01:15:00.255 INFO:teuthology.orchestra.run.smithi096.stdout:2909 write 0x554d6e8 thru 0x555cc2a (0xf543 bytes) 2024-09-22T01:15:00.261 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.261 INFO:teuthology.orchestra.run.smithi096.stdout:2911 write 0x91c8452 thru 0x91cea5e (0x660d bytes) 2024-09-22T01:15:00.266 INFO:teuthology.orchestra.run.smithi096.stdout:2912 writesame 0x4700c65 thru 0x47061f5 data_size 0xd(0x5591 bytes) 2024-09-22T01:15:00.272 INFO:teuthology.orchestra.run.smithi096.stdout:2914 write 0x10800df thru 0x1089b8c (0x9aae bytes) 2024-09-22T01:15:00.274 INFO:teuthology.orchestra.run.smithi096.stdout:2915 write 0xc64be85 thru 0xc65854f (0xc6cb bytes) 2024-09-22T01:15:00.279 INFO:teuthology.orchestra.run.smithi096.stdout:2916 read 0x75541a3 thru 0x75544ec (0x34a bytes) 2024-09-22T01:15:00.279 INFO:teuthology.orchestra.run.smithi096.stdout:2917 writesame 0xc88673e thru 0xc88fa4b data_size 0x7(0x930e bytes) 2024-09-22T01:15:00.288 INFO:teuthology.orchestra.run.smithi096.stdout:2919 trunc from 0xd1dadcc to 0xd3d2977 2024-09-22T01:15:00.298 INFO:teuthology.orchestra.run.smithi096.stdout:2921 punch from 0x71dfc59 to 0x71e26e1, (0x2a88 bytes) 2024-09-22T01:15:00.299 INFO:teuthology.orchestra.run.smithi096.stdout:2922 read 0xc6b5b00 thru 0xc6c245f (0xc960 bytes) 2024-09-22T01:15:00.299 INFO:teuthology.orchestra.run.smithi096.stdout:2923 read 0x762180a thru 0x7622c7e (0x1475 bytes) 2024-09-22T01:15:00.299 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.301 INFO:teuthology.orchestra.run.smithi096.stdout:2927 trunc from 0xd3d2977 to 0xe9a128a 2024-09-22T01:15:00.308 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:00.305+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:00.311 INFO:teuthology.orchestra.run.smithi096.stdout:2928 read 0xce01116 thru 0xce06ea6 (0x5d91 bytes) 2024-09-22T01:15:00.311 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.311 INFO:teuthology.orchestra.run.smithi096.stdout:2932 write 0x20e1257 thru 0x20edf4b (0xccf5 bytes) 2024-09-22T01:15:00.314 INFO:teuthology.orchestra.run.smithi096.stdout:2933 write 0x6a6fba9 thru 0x6a7d807 (0xdc5f bytes) 2024-09-22T01:15:00.318 INFO:teuthology.orchestra.run.smithi096.stdout:2934 write 0x54ee1ee thru 0x54facf0 (0xcb03 bytes) 2024-09-22T01:15:00.324 INFO:teuthology.orchestra.run.smithi096.stdout:2937 read 0xb01e038 thru 0xb023fcc (0x5f95 bytes) 2024-09-22T01:15:00.324 INFO:teuthology.orchestra.run.smithi096.stdout:2938 write 0x25dfad7 thru 0x25e62f9 (0x6823 bytes) 2024-09-22T01:15:00.329 INFO:teuthology.orchestra.run.smithi096.stdout:2939 write 0xe322f5c thru 0xe32a731 (0x77d6 bytes) 2024-09-22T01:15:00.335 INFO:teuthology.orchestra.run.smithi096.stdout:2940 read 0x5d61bda thru 0x5d702d3 (0xe6fa bytes) 2024-09-22T01:15:00.335 INFO:teuthology.orchestra.run.smithi096.stdout:2941 read 0x6f104b0 thru 0x6f1f920 (0xf471 bytes) 2024-09-22T01:15:00.364 INFO:teuthology.orchestra.run.smithi096.stdout:2942 trunc from 0xe9a128a to 0x12f822e 2024-09-22T01:15:00.391 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.391 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.394 INFO:teuthology.orchestra.run.smithi096.stdout:2943 clone 25 order 22 su 1048576 sc 6 2024-09-22T01:15:00.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:00.409 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:00.407+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:00.510 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:00.505+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:00.567 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.567 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:00.612 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:00.607+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:00.662 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:15:00.732 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:00.729+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:00.756 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.756 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:00.777 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone24 from 0x12f822e (overlap 0x36860f) to 0x311240 2024-09-22T01:15:00.824 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:00.821+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:00.890 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-22T01:15:00.908 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:00.908 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:00.914 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:00.909+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:00.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:00.968 INFO:teuthology.orchestra.run.smithi096.stdout:2945 read 0x10d76ec thru 0x10da1ee (0x2b03 bytes) 2024-09-22T01:15:00.970 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:00.970 INFO:teuthology.orchestra.run.smithi096.stdout:2947 punch from 0xc07861 to 0xc09feb, (0x278a bytes) 2024-09-22T01:15:01.000 INFO:teuthology.orchestra.run.smithi096.stdout:2948 write 0x9018178 thru 0x901c6ac (0x4535 bytes) 2024-09-22T01:15:01.006 INFO:teuthology.orchestra.run.smithi096.stdout:2949 punch from 0x834c6b5 to 0x83530c1, (0x6a0c bytes) 2024-09-22T01:15:01.006 INFO:teuthology.orchestra.run.smithi096.stdout:2950 writesame 0x251bf6 thru 0x2579fc data_size 0x1(0x5e07 bytes) 2024-09-22T01:15:01.015 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:01.011+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:01.027 INFO:teuthology.orchestra.run.smithi096.stdout:2951 punch from 0x6689f2 to 0x6739d1, (0xafdf bytes) 2024-09-22T01:15:01.027 INFO:teuthology.orchestra.run.smithi096.stdout:2952 read 0x6ff6d2b thru 0x6ffb865 (0x4b3b bytes) 2024-09-22T01:15:01.027 INFO:teuthology.orchestra.run.smithi096.stdout:2953 write 0x2cff1c7 thru 0x2d02d04 (0x3b3e bytes) 2024-09-22T01:15:01.034 INFO:teuthology.orchestra.run.smithi096.stdout:2954 read 0x2b89794 thru 0x2b9557a (0xbde7 bytes) 2024-09-22T01:15:01.034 INFO:teuthology.orchestra.run.smithi096.stdout:2955 read 0x68cbce4 thru 0x68ccfb9 (0x12d6 bytes) 2024-09-22T01:15:01.034 INFO:teuthology.orchestra.run.smithi096.stdout:2956 read 0x814a6bb thru 0x8155fce (0xb914 bytes) 2024-09-22T01:15:01.034 INFO:teuthology.orchestra.run.smithi096.stdout:2957 read 0x8223139 thru 0x822a0fc (0x6fc4 bytes) 2024-09-22T01:15:01.034 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:01.045 INFO:teuthology.orchestra.run.smithi096.stdout:2959 write 0x990c715 thru 0x99181e7 (0xbad3 bytes) 2024-09-22T01:15:01.050 INFO:teuthology.orchestra.run.smithi096.stdout:2960 read 0x84f80c5 thru 0x84fa97e (0x28ba bytes) 2024-09-22T01:15:01.050 INFO:teuthology.orchestra.run.smithi096.stdout:2961 read 0x178cb8e thru 0x179bc34 (0xf0a7 bytes) 2024-09-22T01:15:01.050 INFO:teuthology.orchestra.run.smithi096.stdout:2962 write 0x59e87e0 thru 0x59f1f39 (0x975a bytes) 2024-09-22T01:15:01.062 INFO:teuthology.orchestra.run.smithi096.stdout:2963 trunc from 0x99181e8 to 0x44274e5 2024-09-22T01:15:01.087 INFO:teuthology.orchestra.run.smithi096.stdout:2964 read 0x37b5a5 thru 0x385492 (0x9eee bytes) 2024-09-22T01:15:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:2965 read 0x2d928ed thru 0x2da25dd (0xfcf1 bytes) 2024-09-22T01:15:01.090 INFO:teuthology.orchestra.run.smithi096.stdout:2966 read 0x2d8ef14 thru 0x2d9475b (0x5848 bytes) 2024-09-22T01:15:01.091 INFO:teuthology.orchestra.run.smithi096.stdout:2968 read 0xab37c8 thru 0xabcc7a (0x94b3 bytes) 2024-09-22T01:15:01.092 INFO:teuthology.orchestra.run.smithi096.stdout:2969 punch from 0x2bf41a6 to 0x2bf8630, (0x448a bytes) 2024-09-22T01:15:01.102 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.102 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.110 INFO:teuthology.orchestra.run.smithi096.stdout:2970 write 0x616f913 thru 0x61786b1 (0x8d9f bytes) 2024-09-22T01:15:01.115 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:01.111+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:01.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:2971 punch from 0x4bb056c to 0x4bb1a4b, (0x14df bytes) 2024-09-22T01:15:01.117 INFO:teuthology.orchestra.run.smithi096.stdout:2972 clone 26 order 18 su 8192 sc 15 2024-09-22T01:15:01.215 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:01.213+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:01.239 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.239 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.254 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:01.317 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:01.311+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:01.415 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.415 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.418 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:01.415+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:01.429 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:01.519 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:01.513+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:01.550 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.550 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.564 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:01.620 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:01.615+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:01.702 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.702 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.718 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:01.761 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:01.759+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:01.822 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:01.819+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:01.823 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone25 from 0x61786b2 (overlap 0x12f822e) to 0x111285c 2024-09-22T01:15:01.897 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:01.898 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:01.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:01.957 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:01.955+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:02.006 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-22T01:15:02.016 INFO:teuthology.orchestra.run.smithi096.stdout:2973 read 0x9b9723 thru 0x9c666f (0xcf4d bytes) 2024-09-22T01:15:02.024 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:02.019+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:02.037 INFO:teuthology.orchestra.run.smithi096.stdout:2976 writesame 0x72bf116 thru 0x72bfa05 data_size 0x10(0x8f0 bytes) 2024-09-22T01:15:02.052 INFO:teuthology.orchestra.run.smithi096.stdout:2977 write 0xa10562a thru 0xa108c7a (0x3651 bytes) 2024-09-22T01:15:02.078 INFO:teuthology.orchestra.run.smithi096.stdout:2978 write 0xd96bda9 thru 0xd976af5 (0xad4d bytes) 2024-09-22T01:15:02.092 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:02.092 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:02.100 INFO:teuthology.orchestra.run.smithi096.stdout:2979 read 0x92bd404 thru 0x92c245f (0x505c bytes) 2024-09-22T01:15:02.100 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:02.111 INFO:teuthology.orchestra.run.smithi096.stdout:2982 write 0xe111034 thru 0xe117068 (0x6035 bytes) 2024-09-22T01:15:02.136 INFO:teuthology.orchestra.run.smithi096.stdout:2984 writesame 0xeb42865 thru 0xeb4667c data_size 0x4(0x3e18 bytes) 2024-09-22T01:15:02.137 INFO:teuthology.orchestra.run.smithi096.stdout:2985 flatten 2024-09-22T01:15:02.149 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:02.147+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:02.156 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:02.226 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:02.221+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:02.292 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:02.292 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:02.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:02.327 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:02.321+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:02.367 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:02.367 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:02.384 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:02.384 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:02.401 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:02.401 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:02.418 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:02.419 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:02.429 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:02.423+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:02.440 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:02.440 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:02.465 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:02.465 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:02.469 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:02.469 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:02.469 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:02.469 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:02.469 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:02.481 INFO:teuthology.orchestra.run.smithi096.stdout:2986 trunc from 0xeb4667d to 0x75c135d 2024-09-22T01:15:02.483 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:02.484 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:02.485 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:02.502 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:02.503 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:02.507 INFO:teuthology.orchestra.run.smithi096.stdout:2988 write 0x313b51f thru 0x31476b9 (0xc19b bytes) 2024-09-22T01:15:02.518 INFO:teuthology.orchestra.run.smithi096.stdout:2989 punch from 0x356a5a to 0x35e9b2, (0x7f58 bytes) 2024-09-22T01:15:02.608 INFO:teuthology.orchestra.run.smithi096.stdout:2990 read 0x6db4edb thru 0x6db5b14 (0xc3a bytes) 2024-09-22T01:15:02.608 INFO:teuthology.orchestra.run.smithi096.stdout:2991 write 0x3da2de4 thru 0x3dab5b5 (0x87d2 bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2992 trunc from 0x75c135d to 0x5887186 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2993 write 0xec13055 thru 0xec1b236 (0x81e2 bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2995 write 0x464e3ec thru 0x4658351 (0x9f66 bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2996 read 0x7f8c035 thru 0x7f98336 (0xc302 bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2997 punch from 0x2494ced to 0x24a2b20, (0xde33 bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:2998 write 0xa008c47 thru 0xa0114e4 (0x889e bytes) 2024-09-22T01:15:02.611 INFO:teuthology.orchestra.run.smithi096.stdout:3001 write 0x31767a7 thru 0x317cff2 (0x684c bytes) 2024-09-22T01:15:02.612 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:02.607+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:02.616 INFO:teuthology.orchestra.run.smithi096.stdout:3005 trunc from 0xec1b237 to 0xbb8303e 2024-09-22T01:15:02.634 INFO:teuthology.orchestra.run.smithi096.stdout:3006 read 0x2c0f7cb thru 0x2c1d73c (0xdf72 bytes) 2024-09-22T01:15:02.652 INFO:teuthology.orchestra.run.smithi096.stdout:3007 trunc from 0xbb8303e to 0x90189d 2024-09-22T01:15:02.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:15:02.712 INFO:teuthology.orchestra.run.smithi096.stdout:3008 writesame 0xb210652 thru 0xb21f023 data_size 0x2(0xe9d2 bytes) 2024-09-22T01:15:02.712 INFO:teuthology.orchestra.run.smithi096.stdout:3009 writesame 0x1a8fd6a thru 0x1a93f26 data_size 0x1(0x41bd bytes) 2024-09-22T01:15:02.713 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:02.709+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:02.729 INFO:teuthology.orchestra.run.smithi096.stdout:3011 write 0x1c4ff7c thru 0x1c535f9 (0x367e bytes) 2024-09-22T01:15:02.733 INFO:teuthology.orchestra.run.smithi096.stdout:3013 read 0x583d0b4 thru 0x584baf0 (0xea3d bytes) 2024-09-22T01:15:02.733 INFO:teuthology.orchestra.run.smithi096.stdout:3015 writesame 0x293172c thru 0x2933662 data_size 0x1(0x1f37 bytes) 2024-09-22T01:15:02.741 INFO:teuthology.orchestra.run.smithi096.stdout:3016 read 0x2c906ff thru 0x2c981e2 (0x7ae4 bytes) 2024-09-22T01:15:02.741 INFO:teuthology.orchestra.run.smithi096.stdout:3017 punch from 0x52401d1 to 0x52465df, (0x640e bytes) 2024-09-22T01:15:02.741 INFO:teuthology.orchestra.run.smithi096.stdout:3018 read 0x9b802f6 thru 0x9b8569e (0x53a9 bytes) 2024-09-22T01:15:02.744 INFO:teuthology.orchestra.run.smithi096.stdout:3019 trunc from 0xb21f024 to 0xd3f4153 2024-09-22T01:15:02.750 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:02.750 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81660 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 897024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5682103329999997, 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063684256999999994, 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.751 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81660 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179638+0000", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6100000000000002e-07 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179640+0000", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5629999999999999e-06 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179646+0000", 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8449999999999999e-06 2024-09-22T01:15:02.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179649+0000", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7669999999999999e-06 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243122+0000", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063473048000000004 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243214+0000", 2024-09-22T01:15:02.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2478000000000005e-05 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243322+0000", 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010829500000000001 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81662 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 905216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5681957019999997, 2024-09-22T01:15:02.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063961117999999997, 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81662 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179653+0000", 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.22e-07 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179654+0000", 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3340000000000001e-06 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179658+0000", 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4340000000000001e-06 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179660+0000", 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5959999999999998e-06 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243370+0000", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063710125000000006 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243487+0000", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116728 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243614+0000", 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012667900000000001 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.759 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81664 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 913408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5681845570000004, 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064198330999999997, 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.760 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81664 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179664+0000", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179665+0000", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1030000000000001e-06 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179669+0000", 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2909999999999999e-06 2024-09-22T01:15:02.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.179671+0000", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5229999999999998e-06 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243664+0000", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0639927 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243753+0000", 2024-09-22T01:15:02.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9555999999999994e-05 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.243862+0000", 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010883 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81791 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 781312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.764 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5587178279999998, 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063538149000000002, 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81791 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189131+0000", 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4900000000000001e-07 2024-09-22T01:15:02.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189133+0000", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.148e-06 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189139+0000", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6060000000000002e-06 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189145+0000", 2024-09-22T01:15:02.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6379999999999998e-06 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252453+0000", 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063307422000000002 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252558+0000", 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010567499999999999 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252669+0000", 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011031099999999999 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81792 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 789504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5586967950000004, 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063793329999999995, 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81792 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:15:02.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189152+0000", 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5699999999999999e-07 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189153+0000", 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.486e-06 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189164+0000", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1012000000000001e-05 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189170+0000", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.378e-06 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252719+0000", 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063549117000000002 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252829+0000", 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109796 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252945+0000", 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000116284 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81793 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5586734079999998, 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064022895999999996, 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81793 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189175+0000", 2024-09-22T01:15:02.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5899999999999998e-07 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189182+0000", 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5130000000000002e-06 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189186+0000", 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1919999999999998e-06 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189189+0000", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6520000000000002e-06 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.252995+0000", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063805929999999997 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253089+0000", 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.4630999999999996e-05 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253198+0000", 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108719 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81794 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 805888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5586558569999998, 2024-09-22T01:15:02.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064298414999999998, 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81794 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:15:02.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189193+0000", 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3099999999999999e-07 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189204+0000", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0740000000000001e-05 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189208+0000", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7729999999999999e-06 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189212+0000", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6559999999999998e-06 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253247+0000", 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064035089000000003 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253364+0000", 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011712700000000001 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253491+0000", 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000126799 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.782 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81795 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 814080~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5586318300000004, 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064526389000000003, 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.783 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81795 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189217+0000", 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1e-07 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189218+0000", 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3430000000000001e-06 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189222+0000", 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4319999999999999e-06 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189224+0000", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253544+0000", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064319935999999994 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253638+0000", 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3812000000000002e-05 2024-09-22T01:15:02.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253743+0000", 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010508100000000001 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81796 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 822272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5586148140000002, 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064799000999999995, 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.787 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81796 2024-09-22T01:15:02.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189234+0000", 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7700000000000001e-07 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189236+0000", 2024-09-22T01:15:02.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6700000000000001e-06 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189245+0000", 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9480000000000003e-06 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189248+0000", 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8619999999999999e-06 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253793+0000", 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064544659000000004 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.253907+0000", 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011349999999999999 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254033+0000", 2024-09-22T01:15:02.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012608500000000001 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81797 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5585630249999998, 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064994761999999998, 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81797 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189285+0000", 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189286+0000", 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3500000000000002e-07 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189288+0000", 2024-09-22T01:15:02.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8080000000000001e-06 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189292+0000", 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7589999999999997e-06 2024-09-22T01:15:02.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189300+0000", 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4669999999999998e-06 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254085+0000", 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064785331000000002 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254177+0000", 2024-09-22T01:15:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1456e-05 2024-09-22T01:15:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254280+0000", 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010350599999999999 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81798 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 838656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5585434469999999, 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065215094000000001, 2024-09-22T01:15:02.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.808 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81798 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:15:02.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189305+0000", 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6199999999999999e-07 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189307+0000", 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5e-06 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189310+0000", 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.563e-06 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189314+0000", 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4340000000000001e-06 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254328+0000", 2024-09-22T01:15:02.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065014409999999995 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254453+0000", 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000124401 2024-09-22T01:15:02.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254520+0000", 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7523999999999999e-05 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.816 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81799 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 846848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:15:02.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5585309440000001, 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065462031000000004, 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81799 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189317+0000", 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189318+0000", 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7099999999999998e-07 2024-09-22T01:15:02.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189319+0000", 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.322e-06 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189322+0000", 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.388e-06 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189325+0000", 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5299999999999999e-06 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254565+0000", 2024-09-22T01:15:02.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065240252999999998 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254664+0000", 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8679999999999997e-05 2024-09-22T01:15:02.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254780+0000", 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115587 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.826 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81800 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 855040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:15:02.827 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5585199970000003, 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065696737000000005, 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.828 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81800 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189328+0000", 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189329+0000", 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1199999999999999e-07 2024-09-22T01:15:02.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189330+0000", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1930000000000001e-06 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189333+0000", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3689999999999998e-06 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189336+0000", 2024-09-22T01:15:02.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254815+0000", 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065479350000000006 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.254911+0000", 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6380999999999998e-05 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255025+0000", 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113835 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81801 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5585028200000002, 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065920638000000004, 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81801 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189346+0000", 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3500000000000002e-07 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189348+0000", 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6670000000000001e-06 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189360+0000", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2582000000000001e-05 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189363+0000", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255063+0000", 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065699750000000001 2024-09-22T01:15:02.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255155+0000", 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2435000000000005e-05 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255266+0000", 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011106 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81802 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5584807769999998, 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066053509999999996, 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81802 2024-09-22T01:15:02.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189368+0000", 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4200000000000002e-07 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189369+0000", 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4130000000000001e-06 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189374+0000", 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2479999999999998e-06 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189376+0000", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5560000000000002e-06 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255307+0000", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.065930585999999999 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255360+0000", 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2852999999999997e-05 2024-09-22T01:15:02.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255421+0000", 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1612000000000006e-05 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81803 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 879616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5584680429999995, 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066282642000000003, 2024-09-22T01:15:02.845 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81803 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189380+0000", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189381+0000", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.17e-07 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189382+0000", 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.187e-06 2024-09-22T01:15:02.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189385+0000", 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4999999999999999e-06 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189389+0000", 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.304e-06 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255467+0000", 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066078908000000006 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255560+0000", 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2607000000000003e-05 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255663+0000", 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010291899999999999 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81804 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5584560940000003, 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066499758000000006, 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81804 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189392+0000", 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189393+0000", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0900000000000001e-07 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189394+0000", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2869999999999999e-06 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189397+0000", 2024-09-22T01:15:02.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4470000000000002e-06 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189400+0000", 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1030000000000002e-06 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255697+0000", 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066297054999999994 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255787+0000", 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9646000000000002e-05 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255892+0000", 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105011 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81805 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 896000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5584445110000003, 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066645572, 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81805 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189404+0000", 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2000000000000001e-07 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189412+0000", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9889999999999992e-06 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189416+0000", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0890000000000002e-06 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189419+0000", 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5720000000000001e-06 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255933+0000", 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066514719999999999 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.255988+0000", 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4400000000000001e-05 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256049+0000", 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1581999999999994e-05 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81806 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 904192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.558424263, 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066870436000000005, 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81806 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189424+0000", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.79e-07 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189426+0000", 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3659999999999999e-06 2024-09-22T01:15:02.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189429+0000", 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5710000000000002e-06 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189436+0000", 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256094+0000", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066657328000000002 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256185+0000", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1278000000000003e-05 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256295+0000", 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109756 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.863 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:81807 3.1e 3:79d90251:::rbd_data.112424dbdb5c.0000000000000000:18 [read 912384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.5584083279999996, 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.067081691999999998, 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:02.864 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 81807 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189440+0000", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3699999999999999e-07 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189442+0000", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4160000000000001e-06 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189447+0000", 2024-09-22T01:15:02.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.916e-06 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.189449+0000", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5409999999999999e-06 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256330+0000", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.066880816999999995 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256418+0000", 2024-09-22T01:15:02.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7715999999999998e-05 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:58.256522+0000", 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010404900000000001 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:02.868 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:02.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:02.871 INFO:teuthology.orchestra.run.smithi096.stdout:3020 read 0xd1790eb thru 0xd1851c9 (0xc0df bytes) 2024-09-22T01:15:02.871 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.871 INFO:teuthology.orchestra.run.smithi096.stdout:3023 punch from 0x54cda22 to 0x54d94a8, (0xba86 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3024 write 0x9107c31 thru 0x910cdf2 (0x51c2 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3025 write 0x11c7120 thru 0x11d4dd1 (0xdcb2 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3026 read 0x32f2bae thru 0x32f2fcb (0x41e bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3027 writesame 0x933f7fd thru 0x934d5ec data_size 0x10(0xddf0 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3030 punch from 0x9fa831b to 0x9fb25bc, (0xa2a1 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3032 trunc from 0xd3f4153 to 0xa776905 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3035 read 0x6574693 thru 0x657c499 (0x7e07 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3038 write 0xe3289fc thru 0xe332d3f (0xa344 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3039 writesame 0xdfb2ad5 thru 0xdfb9c87 data_size 0x1(0x71b3 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3040 punch from 0x6340f91 to 0x6350493, (0xf502 bytes) 2024-09-22T01:15:02.872 INFO:teuthology.orchestra.run.smithi096.stdout:3041 trunc from 0xe332d40 to 0xd3ff40a 2024-09-22T01:15:02.873 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:02.865+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:02.881 INFO:teuthology.orchestra.run.smithi096.stdout:3043 write 0x2f27d9 thru 0x2f2df0 (0x618 bytes) 2024-09-22T01:15:02.882 INFO:teuthology.orchestra.run.smithi096.stdout:3044 read 0x5d42180 thru 0x5d45469 (0x32ea bytes) 2024-09-22T01:15:02.883 INFO:teuthology.orchestra.run.smithi096.stdout:3045 write 0xa8ace2a thru 0xa8b9e50 (0xd027 bytes) 2024-09-22T01:15:02.895 INFO:teuthology.orchestra.run.smithi096.stdout:3046 write 0xbc4232b thru 0xbc4d704 (0xb3da bytes) 2024-09-22T01:15:02.905 INFO:teuthology.orchestra.run.smithi096.stdout:3047 read 0x211044f thru 0x211a7c4 (0xa376 bytes) 2024-09-22T01:15:02.905 INFO:teuthology.orchestra.run.smithi096.stdout:3048 read 0x96a8a75 thru 0x96aada0 (0x232c bytes) 2024-09-22T01:15:02.915 INFO:teuthology.orchestra.run.smithi096.stdout:3049 trunc from 0xd3ff40a to 0x6178123 2024-09-22T01:15:02.939 INFO:teuthology.orchestra.run.smithi096.stdout:3050 write 0x470a5c8 thru 0x47176ad (0xd0e6 bytes) 2024-09-22T01:15:02.952 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:02.952 INFO:teuthology.orchestra.run.smithi096.stdout:3054 read 0x6007ddf thru 0x600e566 (0x6788 bytes) 2024-09-22T01:15:02.953 INFO:teuthology.orchestra.run.smithi096.stdout:3055 write 0x577f7ec thru 0x5783097 (0x38ac bytes) 2024-09-22T01:15:02.970 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:02.965+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:02.979 INFO:teuthology.orchestra.run.smithi096.stdout:3057 write 0xee08529 thru 0xee0cb4c (0x4624 bytes) 2024-09-22T01:15:03.008 INFO:teuthology.orchestra.run.smithi096.stdout:3058 trunc from 0xee0cb4d to 0xd800a9 2024-09-22T01:15:03.008 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:03.009 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:03.009 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.026 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:03.066 INFO:teuthology.orchestra.run.smithi096.stdout:3059 writesame 0xd6b6280 thru 0xd6c5497 data_size 0x8(0xf218 bytes) 2024-09-22T01:15:03.066 INFO:teuthology.orchestra.run.smithi096.stdout:3060 read 0xa78cdc4 thru 0xa795ebe (0x90fb bytes) 2024-09-22T01:15:03.066 INFO:teuthology.orchestra.run.smithi096.stdout:3061 read 0x4ea9b2e thru 0x4eb0232 (0x6705 bytes) 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:3062 punch from 0x5d46e78 to 0x5d4c06e, (0x51f6 bytes) 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:3063 read 0x3af826d thru 0x3afc49e (0x4232 bytes) 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:3064 read 0x697e13d thru 0x698bf18 (0xdddc bytes) 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:3067 punch from 0x7eddabe to 0x7eddee0, (0x422 bytes) 2024-09-22T01:15:03.067 INFO:teuthology.orchestra.run.smithi096.stdout:3068 write 0x19aed18 thru 0x19bdbca (0xeeb3 bytes) 2024-09-22T01:15:03.070 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:03.065+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:03.086 INFO:teuthology.orchestra.run.smithi096.stdout:3070 writesame 0x1487826 thru 0x14933fe data_size 0x1(0xbbd9 bytes) 2024-09-22T01:15:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:3071 read 0xd554414 thru 0xd55ef78 (0xab65 bytes) 2024-09-22T01:15:03.139 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:3073 read 0xb461b1a thru 0xb462f9b (0x1482 bytes) 2024-09-22T01:15:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:3074 punch from 0x44a71cb to 0x44abc09, (0x4a3e bytes) 2024-09-22T01:15:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:3075 punch from 0x274890d to 0x274f6cc, (0x6dbf bytes) 2024-09-22T01:15:03.140 INFO:teuthology.orchestra.run.smithi096.stdout:3076 write 0x291383b thru 0x2923611 (0xfdd7 bytes) 2024-09-22T01:15:03.154 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.154 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:03.154 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:03.154 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:03.154 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.161 INFO:teuthology.orchestra.run.smithi096.stdout:3077 write 0x89fe7dc thru 0x8a09151 (0xa976 bytes) 2024-09-22T01:15:03.171 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:03.172 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:03.165+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:03.180 INFO:teuthology.orchestra.run.smithi096.stdout:3078 writesame 0xa7417c8 thru 0xa743372 data_size 0x3(0x1bab bytes) 2024-09-22T01:15:03.189 INFO:teuthology.orchestra.run.smithi096.stdout:3079 punch from 0x48ea6ca to 0x48eb261, (0xb97 bytes) 2024-09-22T01:15:03.189 INFO:teuthology.orchestra.run.smithi096.stdout:3081 write 0x65092d8 thru 0x6511846 (0x856f bytes) 2024-09-22T01:15:03.208 INFO:teuthology.orchestra.run.smithi096.stdout:3083 punch from 0x9e2a39a to 0x9e376a8, (0xd30e bytes) 2024-09-22T01:15:03.208 INFO:teuthology.orchestra.run.smithi096.stdout:3084 write 0x335bdb5 thru 0x3360d84 (0x4fd0 bytes) 2024-09-22T01:15:03.218 INFO:teuthology.orchestra.run.smithi096.stdout:3085 writesame 0xb5eeffe thru 0xb5fc92e data_size 0x5b(0xd931 bytes) 2024-09-22T01:15:03.231 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:03.231 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:03.231 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-22T01:15:03.231 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:15:03.231 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:15:03.231 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-22T01:15:03.238 INFO:teuthology.orchestra.run.smithi096.stdout:3086 write 0xd3a822c thru 0xd3ab9ac (0x3781 bytes) 2024-09-22T01:15:03.246 INFO:teuthology.orchestra.run.smithi096.stdout:3088 write 0x2e94a24 thru 0x2e9df01 (0x94de bytes) 2024-09-22T01:15:03.257 INFO:teuthology.orchestra.run.smithi096.stdout:3089 write 0x3b29299 thru 0x3b31b33 (0x889b bytes) 2024-09-22T01:15:03.272 INFO:teuthology.orchestra.run.smithi096.stdout:3090 read 0xaf2248d thru 0xaf26a1a (0x458e bytes) 2024-09-22T01:15:03.272 INFO:teuthology.orchestra.run.smithi096.stdout:3091 punch from 0x517ba45 to 0x517d035, (0x15f0 bytes) 2024-09-22T01:15:03.272 INFO:teuthology.orchestra.run.smithi096.stdout:3093 writesame 0x848018e thru 0x848af56 data_size 0x1(0xadc9 bytes) 2024-09-22T01:15:03.281 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:03.279+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992493805999999, 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:03.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:15:03.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:15:03.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992441585999998, 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992241937999999, 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992163235, 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:03.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:03.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:03.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992103241999999, 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:03.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.992030749000001, 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.991962436999998, 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.991904513000001, 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:03.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:03.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.991848623999999, 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.104903847000003, 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:03.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.409536867, 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:15:03.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:15:03.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:03.354 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.017411856000003, 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:03.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.325557608, 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:15:03.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:15:03.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.325439389, 2024-09-22T01:15:03.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.322912448, 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.304466094999999, 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:03.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:03.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:03.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.144839202, 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:15:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.144785107000001, 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:15:03.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:15:03.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.144698811000001, 2024-09-22T01:15:03.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:15:03.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:15:03.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:15:03.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:15:03.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.144069102, 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.403 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi096.stdout:3096 punch from 0x3b49f47 to 0x3b4cf95, (0x304e bytes) 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi096.stdout:3097 read 0x2124acc thru 0x212f5cb (0xab00 bytes) 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi096.stdout:3098 read 0xcc9f9ac thru 0xcca660f (0x6c64 bytes) 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi096.stdout:3099 read 0xa072e9e thru 0xa077f87 (0x50ea bytes) 2024-09-22T01:15:03.407 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3101 trunc from 0xd6c5498 to 0x6c73fbd 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3103 write 0x2a29093 thru 0x2a2fbf3 (0x6b61 bytes) 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3105 punch from 0x49c83ae to 0x49d1ad6, (0x9728 bytes) 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3106 writesame 0xe8653db thru 0xe86f046 data_size 0x47(0x9c6c bytes) 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3107 read 0xc4d84aa thru 0xc4dc288 (0x3ddf bytes) 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:03.408 INFO:teuthology.orchestra.run.smithi096.stdout:3111 write 0x8fdbcb3 thru 0x8fea31c (0xe66a bytes) 2024-09-22T01:15:03.408 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:03.399+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:03.440 INFO:teuthology.orchestra.run.smithi096.stdout:3112 trunc from 0xe86f047 to 0xbd81f04 2024-09-22T01:15:03.451 INFO:teuthology.orchestra.run.smithi096.stdout:3114 punch from 0x6fd2432 to 0x6fda2e3, (0x7eb1 bytes) 2024-09-22T01:15:03.451 INFO:teuthology.orchestra.run.smithi096.stdout:3115 write 0x10083f9 thru 0x10113e3 (0x8feb bytes) 2024-09-22T01:15:03.465 INFO:teuthology.orchestra.run.smithi096.stdout:3119 write 0x7e956bc thru 0x7e989b7 (0x32fc bytes) 2024-09-22T01:15:03.477 INFO:teuthology.orchestra.run.smithi096.stdout:3120 read 0x3904000 thru 0x39118e4 (0xd8e5 bytes) 2024-09-22T01:15:03.478 INFO:teuthology.orchestra.run.smithi096.stdout:3121 write 0xa36f086 thru 0xa379733 (0xa6ae bytes) 2024-09-22T01:15:03.499 INFO:teuthology.orchestra.run.smithi096.stdout:3122 write 0x508f3ba thru 0x5091958 (0x259f bytes) 2024-09-22T01:15:03.506 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:03.503+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:03.506 INFO:teuthology.orchestra.run.smithi096.stdout:3123 clone 27 order 20 su 262144 sc 5 2024-09-22T01:15:03.574 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:03.575 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:03.575 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.589 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:03.630 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:03.627+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:03.693 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:15:03.707 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:03.703+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:03.755 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:03.755 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:03.755 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:03.769 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:03.814 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:03.811+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:03.909 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:03.907+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:03.941 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.710209954, 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:15:03.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:15:03.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:15:03.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:03.947 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.710135125000001, 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:15:03.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.710097101000002, 2024-09-22T01:15:03.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:15:03.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:15:03.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.710047879000001, 2024-09-22T01:15:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:15:03.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.710011532000003, 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:15:03.965 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:15:03.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:15:03.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:03.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.051138641999998, 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:03.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.048957895999997, 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:15:03.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.048885411000001, 2024-09-22T01:15:03.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:15:03.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.048823593999998, 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:15:03.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.645656015, 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:15:03.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968974234000001, 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:15:03.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:03.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968918608999999, 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:03.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:15:04.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968815988999999, 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:15:04.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:04.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968715500999998, 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968654934, 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:15:04.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968597558999999, 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:15:04.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968528848999998, 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968466905, 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.968402938000001, 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:04.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:15:04.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.112307798, 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:04.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:04.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:15:04.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:15:04.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:04.038 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:04.040 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-22T01:15:04.040 INFO:teuthology.orchestra.run.smithi096.stdout:3124 read 0x872eb0a thru 0x873b712 (0xcc09 bytes) 2024-09-22T01:15:04.040 INFO:teuthology.orchestra.run.smithi096.stdout:3125 read 0xa5eedf1 thru 0xa5fa7a5 (0xb9b5 bytes) 2024-09-22T01:15:04.040 INFO:teuthology.orchestra.run.smithi096.stdout:3126 write 0x940789a thru 0x94128bf (0xb026 bytes) 2024-09-22T01:15:04.040 INFO:teuthology.orchestra.run.smithi096.stdout:3127 writesame 0x539681e thru 0x53971cb data_size 0xe(0x9ae bytes) 2024-09-22T01:15:04.041 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:04.035+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:04.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:04.052 INFO:teuthology.orchestra.run.smithi096.stdout:3128 trunc from 0xbd81f04 to 0xe290f16 2024-09-22T01:15:04.061 INFO:teuthology.orchestra.run.smithi096.stdout:3130 write 0x370e747 thru 0x3718177 (0x9a31 bytes) 2024-09-22T01:15:04.069 INFO:teuthology.orchestra.run.smithi096.stdout:3132 write 0xe05c37f thru 0xe06bc98 (0xf91a bytes) 2024-09-22T01:15:04.074 INFO:teuthology.orchestra.run.smithi096.stdout:3133 writesame 0x1a0cadf thru 0x1a1baf3 data_size 0x3(0xf015 bytes) 2024-09-22T01:15:04.096 INFO:teuthology.orchestra.run.smithi096.stdout:3135 writesame 0x7c30ac0 thru 0x7c3e51e data_size 0x1(0xda5f bytes) 2024-09-22T01:15:04.140 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:04.135+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:04.161 INFO:teuthology.orchestra.run.smithi096.stdout:3136 trunc from 0xe290f16 to 0x8bb6936 2024-09-22T01:15:04.179 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:04.179 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:04.179 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:04.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:04.203 INFO:teuthology.orchestra.run.smithi096.stdout:3138 write 0x7d19dd7 thru 0x7d282c4 (0xe4ee bytes) 2024-09-22T01:15:04.212 INFO:teuthology.orchestra.run.smithi096.stdout:3139 read 0x1ef3f0d thru 0x1efe5bf (0xa6b3 bytes) 2024-09-22T01:15:04.212 INFO:teuthology.orchestra.run.smithi096.stdout:3140 punch from 0x3381b2 to 0x33ca5d, (0x48ab bytes) 2024-09-22T01:15:04.212 INFO:teuthology.orchestra.run.smithi096.stdout:3141 write 0x548b201 thru 0x5496069 (0xae69 bytes) 2024-09-22T01:15:04.235 INFO:teuthology.orchestra.run.smithi096.stdout:3142 write 0xdaa48ad thru 0xdaa88a4 (0x3ff8 bytes) 2024-09-22T01:15:04.241 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:04.237+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:04.249 INFO:teuthology.orchestra.run.smithi096.stdout:3143 trunc from 0xdaa88a5 to 0x98de232 2024-09-22T01:15:04.265 INFO:teuthology.orchestra.run.smithi096.stdout:3144 read 0x22d7f2b thru 0x22e6d12 (0xede8 bytes) 2024-09-22T01:15:04.276 INFO:teuthology.orchestra.run.smithi096.stdout:3146 writesame 0xb0ff945 thru 0xb101782 data_size 0x31(0x1e3e bytes) 2024-09-22T01:15:04.276 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:04.277 INFO:teuthology.orchestra.run.smithi096.stdout:3149 punch from 0x62458ea to 0x625562d, (0xfd43 bytes) 2024-09-22T01:15:04.287 INFO:teuthology.orchestra.run.smithi096.stdout:3150 write 0xca1b45d thru 0xca1ecdb (0x387f bytes) 2024-09-22T01:15:04.294 INFO:teuthology.orchestra.run.smithi096.stdout:3151 write 0xbf9f6f5 thru 0xbfac216 (0xcb22 bytes) 2024-09-22T01:15:04.302 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:04.302 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:04.302 INFO:teuthology.orchestra.run.smithi096.stdout:3154 read 0xc2716d8 thru 0xc27fb87 (0xe4b0 bytes) 2024-09-22T01:15:04.302 INFO:teuthology.orchestra.run.smithi096.stdout:3155 read 0xf8d42c thru 0xf95877 (0x844c bytes) 2024-09-22T01:15:04.314 INFO:teuthology.orchestra.run.smithi096.stdout:3156 write 0xdb1f806 thru 0xdb23bfe (0x43f9 bytes) 2024-09-22T01:15:04.321 INFO:teuthology.orchestra.run.smithi096.stdout:3157 writesame 0x81007d2 thru 0x8107c95 data_size 0x35(0x74c4 bytes) 2024-09-22T01:15:04.331 INFO:teuthology.orchestra.run.smithi096.stdout:3158 write 0x52e81b7 thru 0x52f7800 (0xf64a bytes) 2024-09-22T01:15:04.338 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:04.338 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:04.338 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:04.340 INFO:teuthology.orchestra.run.smithi096.stdout:3159 write 0xba61b15 thru 0xba62b51 (0x103d bytes) 2024-09-22T01:15:04.341 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:04.339+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:04.348 INFO:teuthology.orchestra.run.smithi096.stdout:3161 punch from 0x998d9be to 0x999636b, (0x89ad bytes) 2024-09-22T01:15:04.348 INFO:teuthology.orchestra.run.smithi096.stdout:3162 read 0x662d98b thru 0x6635367 (0x79dd bytes) 2024-09-22T01:15:04.348 INFO:teuthology.orchestra.run.smithi096.stdout:3163 writesame 0x19f861d thru 0x19fa5a4 data_size 0x1(0x1f88 bytes) 2024-09-22T01:15:04.354 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:04.362 INFO:teuthology.orchestra.run.smithi096.stdout:3165 writesame 0x512003f thru 0x512074d data_size 0x1(0x70f bytes) 2024-09-22T01:15:04.366 INFO:teuthology.orchestra.run.smithi096.stdout:3166 read 0xa7eb8eb thru 0xa7f3240 (0x7956 bytes) 2024-09-22T01:15:04.367 INFO:teuthology.orchestra.run.smithi096.stdout:3167 write 0x5b03e45 thru 0x5b095c1 (0x577d bytes) 2024-09-22T01:15:04.375 INFO:teuthology.orchestra.run.smithi096.stdout:3168 read 0x2b95e99 thru 0x2b9bdda (0x5f42 bytes) 2024-09-22T01:15:04.376 INFO:teuthology.orchestra.run.smithi096.stdout:3170 clone 28 order 19 su 2048 sc 2 2024-09-22T01:15:04.442 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:04.437+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:04.489 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6542 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.537817938000003, 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012701107999999999, 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6542 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948069+0000", 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948075+0000", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000003e-06 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948086+0000", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0611000000000001e-05 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948098+0000", 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2251e-05 2024-09-22T01:15:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955277+0000", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071787439999999999 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955323+0000", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955513+0000", 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019068700000000001 2024-09-22T01:15:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959026+0000", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035129390000000001 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960685+0000", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016590070000000001 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960726+0000", 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0630999999999998e-05 2024-09-22T01:15:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960770+0000", 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3609000000000002e-05 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6543 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.537758224999997, 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014741681, 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6543 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948129+0000", 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948133+0000", 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980000000000003e-06 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948148+0000", 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4711e-05 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948154+0000", 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3849999999999999e-06 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955772+0000", 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076182760000000002 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955821+0000", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8987999999999999e-05 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.956008+0000", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018672799999999999 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959043+0000", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030352629999999999 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962789+0000", 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037458420000000001 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962826+0000", 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6545999999999999e-05 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962870+0000", 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4118e-05 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7215 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164663603000001, 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012531801, 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7215 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:15:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321239+0000", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5532999999999998e-05 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321246+0000", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321255+0000", 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4180000000000003e-06 2024-09-22T01:15:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333660+0000", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012404851 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333711+0000", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1298999999999998e-05 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333755+0000", 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:15:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7216 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164622753000003, 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670147999999999, 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7216 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321266+0000", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321273+0000", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321277+0000", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7060000000000003e-06 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333792+0000", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012514208000000001 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333862+0000", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0078000000000002e-05 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333934+0000", 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2171999999999999e-05 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7217 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1245184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164602895000002, 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012826168000000001, 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7217 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321286+0000", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.764e-06 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321292+0000", 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.117e-06 2024-09-22T01:15:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321297+0000", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7269999999999998e-06 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333973+0000", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012676481 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334042+0000", 2024-09-22T01:15:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8674e-05 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334110+0000", 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7958000000000002e-05 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7218 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164574494, 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012956225, 2024-09-22T01:15:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7218 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321313+0000", 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:15:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321315+0000", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-06 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321321+0000", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321326+0000", 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.797e-06 2024-09-22T01:15:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334165+0000", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01283831 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334209+0000", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4693000000000001e-05 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334268+0000", 2024-09-22T01:15:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7219 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164552813999997, 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013092405999999999, 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7219 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321336+0000", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8840000000000001e-06 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321342+0000", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321346+0000", 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334311+0000", 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012964361000000001 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334373+0000", 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1791000000000005e-05 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334426+0000", 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3365000000000002e-05 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7220 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1638400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.164533976999998, 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013199765, 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7220 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321353+0000", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.22e-07 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321355+0000", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321362+0000", 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-09-22T01:15:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321376+0000", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3372e-05 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334459+0000", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013083539999999999 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334508+0000", 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8754000000000001e-05 2024-09-22T01:15:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334552+0000", 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7377 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741990971999996, 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013280372, 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7377 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743896+0000", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743899+0000", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6869999999999999e-06 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743912+0000", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3144e-05 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743919+0000", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999994e-06 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747360+0000", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034404280000000001 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747435+0000", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5312e-05 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747724+0000", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028894499999999998 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753696+0000", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059718339999999996 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757027+0000", 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033309030000000001 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757118+0000", 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757176+0000", 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7378 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741959909000002, 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013472072999999999, 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7378 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743927+0000", 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743930+0000", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8559999999999998e-06 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743935+0000", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3199999999999999e-06 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743939+0000", 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3309999999999997e-06 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748128+0000", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041883099999999998 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748217+0000", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9250000000000001e-05 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748581+0000", 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036391 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754767+0000", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061866630000000002 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757274+0000", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025063540000000001 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757344+0000", 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0502999999999999e-05 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757399+0000", 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4242e-05 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7379 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741941695999998, 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013617644999999999, 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7379 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743948+0000", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743956+0000", 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-09-22T01:15:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743961+0000", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.498e-06 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749603+0000", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056418889999999998 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749676+0000", 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2832999999999999e-05 2024-09-22T01:15:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749988+0000", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031244399999999999 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756968+0000", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069804189999999999 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757467+0000", 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049842199999999999 2024-09-22T01:15:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757526+0000", 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9268000000000002e-05 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757562+0000", 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6173000000000001e-05 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7380 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741919934999999, 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013877240000000001, 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7380 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743967+0000", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743970+0000", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8439999999999998e-06 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743977+0000", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743981+0000", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750397+0000", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064165020000000001 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750522+0000", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012448799999999999 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751087+0000", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000564947 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757205+0000", 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061183619999999996 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757713+0000", 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050744799999999997 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757790+0000", 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7334000000000002e-05 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757844+0000", 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3822000000000001e-05 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7381 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741884501000001, 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014017416, 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7381 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744009+0000", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744018+0000", 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0340000000000002e-06 2024-09-22T01:15:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744023+0000", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751817+0000", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077938979999999996 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751909+0000", 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2436999999999996e-05 2024-09-22T01:15:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752214+0000", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030542200000000002 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757581+0000", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053669049999999999 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757913+0000", 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033169200000000002 2024-09-22T01:15:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757965+0000", 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1456999999999999e-05 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758019+0000", 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4775000000000003e-05 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7382 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741856370000001, 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014167732000000001, 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7382 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744033+0000", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744048+0000", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744054+0000", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8159999999999999e-06 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752687+0000", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086338379999999996 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752828+0000", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014017000000000001 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753320+0000", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049253600000000002 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757591+0000", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042712419999999997 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758086+0000", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049453699999999997 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758149+0000", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3261000000000003e-05 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758198+0000", 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7383 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741825980000002, 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016002188000000001, 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7383 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744061+0000", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744063+0000", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744069+0000", 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4829999999999999e-06 2024-09-22T01:15:04.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744073+0000", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753747+0000", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096735599999999995 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753844+0000", 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7019000000000001e-05 2024-09-22T01:15:04.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754147+0000", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030294600000000001 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758259+0000", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004112826 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759947+0000", 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001687287 2024-09-22T01:15:04.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760009+0000", 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2697e-05 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760063+0000", 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3202000000000003e-05 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7384 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741807246, 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016118147999999999, 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7384 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744082+0000", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744087+0000", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744091+0000", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754832+0000", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010740774999999999 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754970+0000", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806199999999999 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.755489+0000", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051922100000000005 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758312+0000", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028228089999999999 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760075+0000", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001762935 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760136+0000", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1231999999999999e-05 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760197+0000", 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1033000000000003e-05 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7385 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.741790272000003, 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017662942000000001, 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.571 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7385 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744099+0000", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744113+0000", 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3722e-05 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744120+0000", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756084+0000", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011963061000000001 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756155+0000", 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1326000000000001e-05 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756461+0000", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030662199999999999 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759930+0000", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034685749999999998 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761621+0000", 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001691173 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761703+0000", 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1729000000000003e-05 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761759+0000", 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6125999999999998e-05 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:19714 3.1d e46/28 3:bd0a6cd9:::rbd_data.11247e2569dd.0000000000000006:head v 46'43, mlcod=41'41)", 2024-09-22T01:15:04.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.277473366999999, 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017168761000000001, 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:04.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208415+0000", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9429999999999999e-06 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211325+0000", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029099019999999998 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211356+0000", 2024-09-22T01:15:04.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1590000000000001e-05 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211371+0000", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4678000000000001e-05 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211454+0000", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2708999999999997e-05 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211480+0000", 2024-09-22T01:15:04.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6264000000000001e-05 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225527+0000", 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014047074 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225582+0000", 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4601000000000001e-05 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22366 3.c 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.200808664999997, 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022689654, 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22366 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285080+0000", 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.781e-06 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289468+0000", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043884040000000003 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289523+0000", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5278999999999997e-05 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289551+0000", 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7574999999999999e-05 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289705+0000", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015408899999999999 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289922+0000", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021745600000000001 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.290362+0000", 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043983999999999999 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305662+0000", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015299975 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307559+0000", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018972259999999999 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307658+0000", 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8330000000000002e-05 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307767+0000", 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010969899999999999 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:29506 3.e e61/58 3:751da8e3:::rbd_data.11242d507bd4.0000000000000006:head v 61'197, mlcod=59'195)", 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.625179318000001, 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017335876, 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:04.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860710+0000", 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4830000000000002e-06 2024-09-22T01:15:04.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865107+0000", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043974649999999997 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865146+0000", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9214000000000001e-05 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865166+0000", 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:15:04.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865300+0000", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134039 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865338+0000", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7796000000000002e-05 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878002+0000", 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012663855999999999 2024-09-22T01:15:04.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878043+0000", 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1433999999999997e-05 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:04.590 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:04.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:04.592 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:04.587+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:04.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:04.693 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:04.687+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:04.778 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone27 from 0xdb23bff (overlap 0x8bb6936) to 0x37e2a15 2024-09-22T01:15:04.779 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:04.779 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:04.779 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:04.779 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:04.793 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:04.791+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:04.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:04.815 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:04.816 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:04.834 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:04.834 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:04.895 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:04.891+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:04.909 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:04.909 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:04.935 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:04.935 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:04.941 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:04.941 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:04.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:04.961 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:04.997 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:04.997 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:04.993+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:05.027 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:05.028 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:05.052 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:05.053 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:05.069 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-22T01:15:05.089 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.090 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.096 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:05.093+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:05.107 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:05.196 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:05.193+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:05.293 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.294 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.297 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:05.294+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:05.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:05.398 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:05.394+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:05.471 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.471 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.476 INFO:teuthology.orchestra.run.smithi096.stdout:3172 write 0x2bffe25 thru 0x2c09e44 (0xa020 bytes) 2024-09-22T01:15:05.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:05.499 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:05.494+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:05.510 INFO:teuthology.orchestra.run.smithi096.stdout:3173 read 0x94dc20 thru 0x959c2f (0xc010 bytes) 2024-09-22T01:15:05.511 INFO:teuthology.orchestra.run.smithi096.stdout:3174 punch from 0x83f0697 to 0x83f2000, (0x1969 bytes) 2024-09-22T01:15:05.517 INFO:teuthology.orchestra.run.smithi096.stdout:3179 writesame 0xdd0ce7a thru 0xdd14f27 data_size 0x5b(0x80ae bytes) 2024-09-22T01:15:05.518 INFO:teuthology.orchestra.run.smithi096.stdout:3181 read 0xd4be253 thru 0xd4c8cde (0xaa8c bytes) 2024-09-22T01:15:05.518 INFO:teuthology.orchestra.run.smithi096.stdout:3182 write 0x155a64c thru 0x155b892 (0x1247 bytes) 2024-09-22T01:15:05.547 INFO:teuthology.orchestra.run.smithi096.stdout:3183 trunc from 0xdd14f28 to 0x9d5754e 2024-09-22T01:15:05.569 INFO:teuthology.orchestra.run.smithi096.stdout:3184 writesame 0xa01b566 thru 0xa02475d data_size 0x6c(0x91f8 bytes) 2024-09-22T01:15:05.584 INFO:teuthology.orchestra.run.smithi096.stdout:3185 trunc from 0xa02475e to 0xa6f2c 2024-09-22T01:15:05.599 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:05.597+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:05.632 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:05.633 INFO:teuthology.orchestra.run.smithi096.stdout:3187 read 0x7c152 thru 0x89655 (0xd504 bytes) 2024-09-22T01:15:05.634 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.634 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.636 INFO:teuthology.orchestra.run.smithi096.stdout:3189 read 0x3cb0d thru 0x3ff8f (0x3483 bytes) 2024-09-22T01:15:05.646 INFO:teuthology.orchestra.run.smithi096.stdout:3190 write 0x726a8c thru 0x72a877 (0x3dec bytes) 2024-09-22T01:15:05.650 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:05.666 INFO:teuthology.orchestra.run.smithi096.stdout:3191 writesame 0x48976fa thru 0x48a02a4 data_size 0x1(0x8bab bytes) 2024-09-22T01:15:05.676 INFO:teuthology.orchestra.run.smithi096.stdout:3193 write 0x6739f0b thru 0x673df20 (0x4016 bytes) 2024-09-22T01:15:05.684 INFO:teuthology.orchestra.run.smithi096.stdout:3194 read 0x3a775b0 thru 0x3a7be9f (0x48f0 bytes) 2024-09-22T01:15:05.700 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:05.697+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:05.703 INFO:teuthology.orchestra.run.smithi096.stdout:3195 write 0xc88d05b thru 0xc88d9ed (0x993 bytes) 2024-09-22T01:15:05.720 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:15:05.726 INFO:teuthology.orchestra.run.smithi096.stdout:3196 trunc from 0xc88d9ee to 0xbbba53 2024-09-22T01:15:05.744 INFO:teuthology.orchestra.run.smithi096.stdout:3197 read 0x4ceca2 thru 0x4da966 (0xbcc5 bytes) 2024-09-22T01:15:05.744 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:05.744 INFO:teuthology.orchestra.run.smithi096.stdout:3200 punch from 0x36aa03 to 0x377b80, (0xd17d bytes) 2024-09-22T01:15:05.757 INFO:teuthology.orchestra.run.smithi096.stdout:3201 write 0x4ac5f14 thru 0x4ad25e6 (0xc6d3 bytes) 2024-09-22T01:15:05.767 INFO:teuthology.orchestra.run.smithi096.stdout:3204 trunc from 0x4ad25e7 to 0x5fa2a70 2024-09-22T01:15:05.777 INFO:teuthology.orchestra.run.smithi096.stdout:3209 trunc from 0x5fa2a70 to 0x7173325 2024-09-22T01:15:05.785 INFO:teuthology.orchestra.run.smithi096.stdout:3210 read 0x55181e7 thru 0x5522a3f (0xa859 bytes) 2024-09-22T01:15:05.785 INFO:teuthology.orchestra.run.smithi096.stdout:3211 punch from 0x6c197d2 to 0x6c1de4a, (0x4678 bytes) 2024-09-22T01:15:05.785 INFO:teuthology.orchestra.run.smithi096.stdout:3213 writesame 0x686dd6a thru 0x6878fcd data_size 0x1c(0xb264 bytes) 2024-09-22T01:15:05.801 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:05.798+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:05.803 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.803 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.808 INFO:teuthology.orchestra.run.smithi096.stdout:3214 write 0xa5aba68 thru 0xa5ac05f (0x5f8 bytes) 2024-09-22T01:15:05.816 INFO:teuthology.orchestra.run.smithi096.stdout:3215 read 0x2c49450 thru 0x2c4e397 (0x4f48 bytes) 2024-09-22T01:15:05.816 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:05.816 INFO:teuthology.orchestra.run.smithi096.stdout:3220 write 0x560e2fc thru 0x5615bf0 (0x78f5 bytes) 2024-09-22T01:15:05.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:05.825 INFO:teuthology.orchestra.run.smithi096.stdout:3221 trunc from 0xa5ac060 to 0xb584d1a 2024-09-22T01:15:05.833 INFO:teuthology.orchestra.run.smithi096.stdout:3222 read 0x697e1f0 thru 0x698b12b (0xcf3c bytes) 2024-09-22T01:15:05.833 INFO:teuthology.orchestra.run.smithi096.stdout:3223 writesame 0x533c2c2 thru 0x5347b38 data_size 0xb(0xb877 bytes) 2024-09-22T01:15:05.851 INFO:teuthology.orchestra.run.smithi096.stdout:3225 trunc from 0xb584d1a to 0x4a17d18 2024-09-22T01:15:05.902 INFO:teuthology.orchestra.run.smithi096.stdout:3226 trunc from 0x4a17d18 to 0xe267db 2024-09-22T01:15:05.902 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:05.898+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:05.946 INFO:teuthology.orchestra.run.smithi096.stdout:3227 write 0xe9e0079 thru 0xe9edf91 (0xdf19 bytes) 2024-09-22T01:15:05.956 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:05.956 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:05.971 INFO:teuthology.orchestra.run.smithi096.stdout:3228 trunc from 0xe9edf92 to 0x321db26 2024-09-22T01:15:05.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:05.989 INFO:teuthology.orchestra.run.smithi096.stdout:3229 write 0x28e67da thru 0x28e96c5 (0x2eec bytes) 2024-09-22T01:15:05.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:05.999 INFO:teuthology.orchestra.run.smithi096.stdout:3231 read 0xd176f7 thru 0xd21547 (0x9e51 bytes) 2024-09-22T01:15:05.999 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:06.003 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:05.997+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:06.021 INFO:teuthology.orchestra.run.smithi096.stdout:3233 writesame 0xc33e6ff thru 0xc33ff01 data_size 0x3(0x1803 bytes) 2024-09-22T01:15:06.021 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:06.033 INFO:teuthology.orchestra.run.smithi096.stdout:3235 write 0xe6c296c thru 0xe6cdb28 (0xb1bd bytes) 2024-09-22T01:15:06.041 INFO:teuthology.orchestra.run.smithi096.stdout:3236 write 0x6e56b34 thru 0x6e65294 (0xe761 bytes) 2024-09-22T01:15:06.049 INFO:teuthology.orchestra.run.smithi096.stdout:3237 write 0x7d1eac6 thru 0x7d291a4 (0xa6df bytes) 2024-09-22T01:15:06.057 INFO:teuthology.orchestra.run.smithi096.stdout:3239 write 0xaad7401 thru 0xaadc7cc (0x53cc bytes) 2024-09-22T01:15:06.064 INFO:teuthology.orchestra.run.smithi096.stdout:3240 write 0x611558e thru 0x6123aef (0xe562 bytes) 2024-09-22T01:15:06.076 INFO:teuthology.orchestra.run.smithi096.stdout:3241 read 0x67bbd31 thru 0x67c5552 (0x9822 bytes) 2024-09-22T01:15:06.076 INFO:teuthology.orchestra.run.smithi096.stdout:3242 write 0x3b8354 thru 0x3c191a (0x95c7 bytes) 2024-09-22T01:15:06.092 INFO:teuthology.orchestra.run.smithi096.stdout:3243 writesame 0xe898768 thru 0xe8a71af data_size 0x62(0xea48 bytes) 2024-09-22T01:15:06.092 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:06.093 INFO:teuthology.orchestra.run.smithi096.stdout:3247 write 0xe1e7934 thru 0xe1ec396 (0x4a63 bytes) 2024-09-22T01:15:06.103 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:06.104 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:06.101+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:06.120 INFO:teuthology.orchestra.run.smithi096.stdout:3249 trunc from 0xe8a71b0 to 0x4b62917 2024-09-22T01:15:06.133 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.133 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.139 INFO:teuthology.orchestra.run.smithi096.stdout:3250 clone 29 order 19 su 4096 sc 8 2024-09-22T01:15:06.148 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:06.205 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:06.201+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:06.281 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.282 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.295 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:06.341 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:06.338+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:06.407 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:06.402+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:06.413 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.413 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:06.514 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:06.510+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:06.592 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.592 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:06.609 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:06.605+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:06.700 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:15:06.709 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:06.706+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:06.773 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.773 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:06.811 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:06.805+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:06.822 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone28 from 0x4b62917 (overlap 0xa6f2c) to 0x53796 2024-09-22T01:15:06.911 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:06.906+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:06.951 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:06.951 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:06.964 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-22T01:15:06.967 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:07.053 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:07.050+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:07.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:07.124 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:07.121+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:07.138 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:07.138 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:07.152 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:07.169 INFO:teuthology.orchestra.run.smithi096.stdout:3251 write 0xb0b0887 thru 0xb0bfcb4 (0xf42e bytes) 2024-09-22T01:15:07.192 INFO:teuthology.orchestra.run.smithi096.stdout:3252 trunc from 0xb0bfcb5 to 0x6b9a1d1 2024-09-22T01:15:07.207 INFO:teuthology.orchestra.run.smithi096.stdout:3253 write 0x2e1c2b8 thru 0x2e26d44 (0xaa8d bytes) 2024-09-22T01:15:07.226 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:07.226 INFO:teuthology.orchestra.run.smithi096.stdout:3257 read 0xbaebb4 thru 0xbb45ae (0x59fb bytes) 2024-09-22T01:15:07.226 INFO:teuthology.orchestra.run.smithi096.stdout:3259 read 0x41c6500 thru 0x41d02d9 (0x9dda bytes) 2024-09-22T01:15:07.227 INFO:teuthology.orchestra.run.smithi096.stdout:3260 punch from 0x630d166 to 0x63112f4, (0x418e bytes) 2024-09-22T01:15:07.227 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:07.228 INFO:teuthology.orchestra.run.smithi096.stdout:3262 trunc from 0x6b9a1d1 to 0x597a5c6 2024-09-22T01:15:07.231 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:07.231 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:07.241 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:07.238+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:07.242 INFO:teuthology.orchestra.run.smithi096.stdout:3263 writesame 0x6f5d30f thru 0x6f5faa1 data_size 0xb(0x2793 bytes) 2024-09-22T01:15:07.242 INFO:teuthology.orchestra.run.smithi096.stdout:3264 punch from 0x6d81292 to 0x6d891c5, (0x7f33 bytes) 2024-09-22T01:15:07.243 INFO:teuthology.orchestra.run.smithi096.stdout:3265 read 0x8ecee8 thru 0x8f1345 (0x445e bytes) 2024-09-22T01:15:07.250 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:07.250 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:07.256 INFO:teuthology.orchestra.run.smithi096.stdout:3267 write 0xc8ec599 thru 0xc8f4e9a (0x8902 bytes) 2024-09-22T01:15:07.268 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:07.269 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:07.270 INFO:teuthology.orchestra.run.smithi096.stdout:3268 writesame 0x28ade18 thru 0x28ba539 data_size 0x47(0xc722 bytes) 2024-09-22T01:15:07.286 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:07.286 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:07.305 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:07.305 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:07.314 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:07.309+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:07.326 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:07.326 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:07.342 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:07.343 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:07.345 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:07.345 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:07.352 INFO:teuthology.orchestra.run.smithi096.stdout:3270 trunc from 0xc8f4e9b to 0x69c18e2 2024-09-22T01:15:07.358 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:07.365 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:07.365 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:07.367 INFO:teuthology.orchestra.run.smithi096.stdout:3272 writesame 0x259a7a3 thru 0x25a33bf data_size 0x1(0x8c1d bytes) 2024-09-22T01:15:07.399 INFO:teuthology.orchestra.run.smithi096.stdout:3273 trunc from 0x69c18e2 to 0x51f9d20 2024-09-22T01:15:07.407 INFO:teuthology.orchestra.run.smithi096.stdout:3274 read 0x77ebb8 thru 0x7874c6 (0x890f bytes) 2024-09-22T01:15:07.408 INFO:teuthology.orchestra.run.smithi096.stdout:3275 punch from 0x14b652b to 0x14be593, (0x8068 bytes) 2024-09-22T01:15:07.411 INFO:teuthology.orchestra.run.smithi096.stdout:3276 trunc from 0x51f9d20 to 0x36ffac4 2024-09-22T01:15:07.415 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:07.409+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:07.437 INFO:teuthology.orchestra.run.smithi096.stdout:3277 punch from 0x259bec6 to 0x259f594, (0x36ce bytes) 2024-09-22T01:15:07.449 INFO:teuthology.orchestra.run.smithi096.stdout:3278 write 0x6d2462c thru 0x6d28e96 (0x486b bytes) 2024-09-22T01:15:07.462 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:07.462 INFO:teuthology.orchestra.run.smithi096.stdout:3280 writesame 0x22a7813 thru 0x22a9dba data_size 0x14(0x25a8 bytes) 2024-09-22T01:15:07.470 INFO:teuthology.orchestra.run.smithi096.stdout:3281 punch from 0x122c9db to 0x1230320, (0x3945 bytes) 2024-09-22T01:15:07.470 INFO:teuthology.orchestra.run.smithi096.stdout:3282 read 0x5e4a2f6 thru 0x5e51fa4 (0x7caf bytes) 2024-09-22T01:15:07.475 INFO:teuthology.orchestra.run.smithi096.stdout:3283 trunc from 0x6d28e97 to 0x3c90b5f 2024-09-22T01:15:07.491 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:07.491 INFO:teuthology.orchestra.run.smithi096.stdout:3286 write 0x36a822d thru 0x36b5fa9 (0xdd7d bytes) 2024-09-22T01:15:07.511 INFO:teuthology.orchestra.run.smithi096.stdout:3287 trunc from 0x3c90b5f to 0xb78947f 2024-09-22T01:15:07.515 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:07.513+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:07.520 INFO:teuthology.orchestra.run.smithi096.stdout:3288 writesame 0x41c03ae thru 0x41ca3e1 data_size 0x1(0xa034 bytes) 2024-09-22T01:15:07.529 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:07.529 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:07.529 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:07.544 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:07.579 INFO:teuthology.orchestra.run.smithi096.stdout:3292 clone 30 order 18 su 131072 sc 12 2024-09-22T01:15:07.616 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:07.613+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7225 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.664 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.348247659000002, 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015286766, 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7225 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:07.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312788+0000", 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312789+0000", 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0839999999999999e-06 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312794+0000", 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0699999999999997e-06 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312811+0000", 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7129000000000002e-05 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312823+0000", 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2217e-05 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327585+0000", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014762143 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.327837+0000", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025185799999999999 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328075+0000", 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023726500000000001 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.668 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.348195439000001, 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:15:07.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347995791000002, 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.673 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:07.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347917088000003, 2024-09-22T01:15:07.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:07.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.681 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347857095000002, 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.682 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:07.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:07.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:07.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347784601999997, 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.686 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:07.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:07.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347716290000001, 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:07.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:07.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347658365999997, 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 51.347602477000002, 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.699 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:07.703 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.460657699999999, 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:07.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.765290720000003, 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.708 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:15:07.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:15:07.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:15:07.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.373165708999998, 2024-09-22T01:15:07.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:07.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.681311461, 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:15:07.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:15:07.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.681193241999999, 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:15:07.723 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:15:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:15:07.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:15:07.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:15:07.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.678666301, 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.729 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:15:07.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:15:07.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.660219947999998, 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.735 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:07.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:07.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:07.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24022 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:07.740 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.500593055, 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015226853, 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24022 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160442+0000", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160443+0000", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5000000000000001e-07 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160449+0000", 2024-09-22T01:15:07.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6930000000000004e-06 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160465+0000", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6115000000000001e-05 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160478+0000", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3266e-05 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164693+0000", 2024-09-22T01:15:07.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.004214364 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.164957+0000", 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026397900000000001 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.166000+0000", 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0010431660000000001 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169422+0000", 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034219680000000001 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175400+0000", 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059782790000000004 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175551+0000", 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000151099 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175669+0000", 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011797400000000001 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.50053896, 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.746 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:15:07.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:15:07.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.500452664000001, 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.752 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:07.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.757 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.499822954999999, 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:07.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:07.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:07.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.763 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.764 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:07.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:07.765 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:07.758+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:07.865 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:07.862+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:07.902 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone29 from 0xb78947f (overlap 0x36ffac4) to 0x15ffde8 2024-09-22T01:15:07.945 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:07.945 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:102204 3.b 3:d073232a:::rbd_data.1124ee2fa883.000000000000004b:head [stat out=16b,delete] snapc 20=[20] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:07.939648+0000", 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0052109779999999998, 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052308600000000004, 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for sub ops", 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 102204 2024-09-22T01:15:07.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939648+0000", 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939648+0000", 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939651+0000", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8700000000000001e-06 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939662+0000", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1646e-05 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939701+0000", 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8711999999999998e-05 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939723+0000", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1942000000000001e-05 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.939867+0000", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143444 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.940096+0000", 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000229418 2024-09-22T01:15:07.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.940649+0000", 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055303199999999996 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944046+0000", 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033965430000000001 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:102205 3.8 e76/24 3:13d290f1:::rbd_data.1124ee2fa883.000000000000004c:head v 76'233, mlcod=75'227)", 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:07.942639+0000", 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.002219651, 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002306088, 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942639+0000", 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942639+0000", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942641+0000", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4500000000000001e-06 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942653+0000", 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1843e-05 2024-09-22T01:15:07.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942678+0000", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5536000000000001e-05 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942723+0000", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4708e-05 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942811+0000", 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7719999999999994e-05 2024-09-22T01:15:07.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942831+0000", 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0373999999999998e-05 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:102209 3.c e76/24 3:353ecd0f:::rbd_data.1124ee2fa883.0000000000000050:head v 76'327, mlcod=76'324)", 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:07.942849+0000", 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.0020102359999999999, 2024-09-22T01:15:07.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002160679, 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942849+0000", 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942849+0000", 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942850+0000", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.254e-06 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942860+0000", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7829999999999993e-06 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942879+0000", 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9729999999999999e-05 2024-09-22T01:15:07.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.942893+0000", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3346000000000001e-05 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.943004+0000", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111282 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.943061+0000", 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6993999999999997e-05 2024-09-22T01:15:07.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop_reply(client.4388.0:102204 3.b e76/24 ondisk, result = 0)", 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:07.944697+0000", 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.000161516, 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036925400000000001, 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944697+0000", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944697+0000", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944699+0000", 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3680000000000001e-06 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944708+0000", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0469999999999999e-06 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944730+0000", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2129000000000001e-05 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944745+0000", 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4704e-05 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944865+0000", 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120321 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:07.944884+0000", 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9449e-05 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 4 2024-09-22T01:15:07.961 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:07.966 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:07.962+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:07.966 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:08.067 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:08.062+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:08.133 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:08.133 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:08.133 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:08.147 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:08.161 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-22T01:15:08.185 INFO:teuthology.orchestra.run.smithi096.stdout:3293 writesame 0xd831954 thru 0xd83c3be data_size 0x1(0xaa6b bytes) 2024-09-22T01:15:08.193 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:08.190+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:08.203 INFO:teuthology.orchestra.run.smithi096.stdout:3294 trunc from 0xd83c3bf to 0x4558d16 2024-09-22T01:15:08.249 INFO:teuthology.orchestra.run.smithi096.stdout:3296 write 0xba387f1 thru 0xba3f8a9 (0x70b9 bytes) 2024-09-22T01:15:08.269 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:08.265+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:08.270 INFO:teuthology.orchestra.run.smithi096.stdout:3297 write 0xd94f0a7 thru 0xd952fa4 (0x3efe bytes) 2024-09-22T01:15:08.287 INFO:teuthology.orchestra.run.smithi096.stdout:3298 write 0xdf1e2ed thru 0xdf2de3a (0xfb4e bytes) 2024-09-22T01:15:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:3299 read 0x98e9d8a thru 0x98ecdad (0x3024 bytes) 2024-09-22T01:15:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:3300 read 0xdafcd7 thru 0xdb5d96 (0x60c0 bytes) 2024-09-22T01:15:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:3301 read 0xd4fd9cb thru 0xd506d13 (0x9349 bytes) 2024-09-22T01:15:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:3302 punch from 0x10172d to 0x1019e8, (0x2bb bytes) 2024-09-22T01:15:08.296 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:08.297 INFO:teuthology.orchestra.run.smithi096.stdout:3304 read 0x7878986 thru 0x78805cf (0x7c4a bytes) 2024-09-22T01:15:08.297 INFO:teuthology.orchestra.run.smithi096.stdout:3305 write 0x23a1d5f thru 0x23addcf (0xc071 bytes) 2024-09-22T01:15:08.306 INFO:teuthology.orchestra.run.smithi096.stdout:3306 write 0x9731664 thru 0x9732c74 (0x1611 bytes) 2024-09-22T01:15:08.313 INFO:teuthology.orchestra.run.smithi096.stdout:3307 read 0x7c33c8c thru 0x7c3fba8 (0xbf1d bytes) 2024-09-22T01:15:08.313 INFO:teuthology.orchestra.run.smithi096.stdout:3309 punch from 0x90adf14 to 0x90b834f, (0xa43b bytes) 2024-09-22T01:15:08.314 INFO:teuthology.orchestra.run.smithi096.stdout:3310 writesame 0x58cf71c thru 0x58dc11e data_size 0x1(0xca03 bytes) 2024-09-22T01:15:08.323 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.093755113, 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:15:08.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:15:08.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.093680284000001, 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.093642260000003, 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.339 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:08.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:15:08.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:15:08.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:08.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.093593038000002, 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:08.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.093556691000003, 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.351 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:15:08.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.434683800999998, 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:15:08.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:15:08.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.432503054999998, 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:08.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:15:08.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.432430570000001, 2024-09-22T01:15:08.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:15:08.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:15:08.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.432368752999999, 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:15:08.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:15:08.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.029201174000001, 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.374 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:08.378 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352519393000001, 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:08.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:15:08.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352463768, 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:15:08.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352361148, 2024-09-22T01:15:08.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:08.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:15:08.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352260659999999, 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:15:08.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352200093, 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352142718000003, 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:08.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352074008000002, 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:15:08.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.352012064, 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:15:08.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:15:08.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.351948096999998, 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:15:08.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:15:08.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 24.495852957, 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:15:08.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:15:08.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:08.424 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:08.425 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:08.425 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:08.425 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:15:08.425 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:15:08.427 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:08.422+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:08.427 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3311 read 0xc9245f9 thru 0xc92890c (0x4314 bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3312 read 0x546f357 thru 0x547a159 (0xae03 bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3314 read 0xa0f5c96 thru 0xa0fca31 (0x6d9c bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3315 read 0x7aca51f thru 0x7ad0b46 (0x6628 bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3316 read 0xae55039 thru 0xae55e4e (0xe16 bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3317 punch from 0x35a5e99 to 0x35af305, (0x946c bytes) 2024-09-22T01:15:08.428 INFO:teuthology.orchestra.run.smithi096.stdout:3318 read 0x9b9ae58 thru 0x9b9bcc6 (0xe6f bytes) 2024-09-22T01:15:08.429 INFO:teuthology.orchestra.run.smithi096.stdout:3321 writesame 0x3a91b6e thru 0x3a9dbe2 data_size 0xb(0xc075 bytes) 2024-09-22T01:15:08.429 INFO:teuthology.orchestra.run.smithi096.stdout:3323 read 0x2ab1e5f thru 0x2ab66e3 (0x4885 bytes) 2024-09-22T01:15:08.429 INFO:teuthology.orchestra.run.smithi096.stdout:3325 read 0x71cb731 thru 0x71d7ba1 (0xc471 bytes) 2024-09-22T01:15:08.429 INFO:teuthology.orchestra.run.smithi096.stdout:3326 punch from 0x5f1f2cb to 0x5f27a66, (0x879b bytes) 2024-09-22T01:15:08.429 INFO:teuthology.orchestra.run.smithi096.stdout:3327 trunc from 0xdf2de3b to 0xeb8f57d 2024-09-22T01:15:08.432 INFO:teuthology.orchestra.run.smithi096.stdout:3328 writesame 0x487c694 thru 0x48872a4 data_size 0x1(0xac11 bytes) 2024-09-22T01:15:08.453 INFO:teuthology.orchestra.run.smithi096.stdout:3329 trunc from 0xeb8f57d to 0x5ff0890 2024-09-22T01:15:08.472 INFO:teuthology.orchestra.run.smithi096.stdout:3330 read 0x4dfc6 thru 0x52c25 (0x4c60 bytes) 2024-09-22T01:15:08.489 INFO:teuthology.orchestra.run.smithi096.stdout:3331 trunc from 0x5ff0890 to 0xb7b2fe9 2024-09-22T01:15:08.498 INFO:teuthology.orchestra.run.smithi096.stdout:3333 write 0x5815940 thru 0x5817e70 (0x2531 bytes) 2024-09-22T01:15:08.502 INFO:teuthology.orchestra.run.smithi096.stdout:3334 read 0x14c3805 thru 0x14c7ddb (0x45d7 bytes) 2024-09-22T01:15:08.509 INFO:teuthology.orchestra.run.smithi096.stdout:3336 trunc from 0xb7b2fe9 to 0x70e5fb5 2024-09-22T01:15:08.518 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:08.527 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:08.522+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:08.532 INFO:teuthology.orchestra.run.smithi096.stdout:3338 write 0x96598c6 thru 0x965cfd2 (0x370d bytes) 2024-09-22T01:15:08.549 INFO:teuthology.orchestra.run.smithi096.stdout:3340 trunc from 0x965cfd3 to 0x35a7b01 2024-09-22T01:15:08.574 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:08.574 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:08.575 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:08.575 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:08.582 INFO:teuthology.orchestra.run.smithi096.stdout:3341 punch from 0x1085f3 to 0x1103b5, (0x7dc2 bytes) 2024-09-22T01:15:08.582 INFO:teuthology.orchestra.run.smithi096.stdout:3343 write 0x7a734a thru 0x7afb57 (0x880e bytes) 2024-09-22T01:15:08.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:08.618 INFO:teuthology.orchestra.run.smithi096.stdout:3346 write 0xd3e4673 thru 0xd3f1558 (0xcee6 bytes) 2024-09-22T01:15:08.622 INFO:teuthology.orchestra.run.smithi096.stdout:3350 read 0xc2cdb20 thru 0xc2cfdcc (0x22ad bytes) 2024-09-22T01:15:08.622 INFO:teuthology.orchestra.run.smithi096.stdout:3351 write 0x409db33 thru 0x40a40f4 (0x65c2 bytes) 2024-09-22T01:15:08.627 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:08.621+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:08.627 INFO:teuthology.orchestra.run.smithi096.stdout:3353 punch from 0xb244705 to 0xb251501, (0xcdfc bytes) 2024-09-22T01:15:08.640 INFO:teuthology.orchestra.run.smithi096.stdout:3354 trunc from 0xd3f1559 to 0x49e4654 2024-09-22T01:15:08.652 INFO:teuthology.orchestra.run.smithi096.stdout:3355 write 0x2a9801e thru 0x2aa1311 (0x92f4 bytes) 2024-09-22T01:15:08.661 INFO:teuthology.orchestra.run.smithi096.stdout:3357 trunc from 0x49e4654 to 0x55bfd23 2024-09-22T01:15:08.670 INFO:teuthology.orchestra.run.smithi096.stdout:3358 trunc from 0x55bfd23 to 0x63cec51 2024-09-22T01:15:08.676 INFO:teuthology.orchestra.run.smithi096.stdout:3359 punch from 0xe41360 to 0xe482c3, (0x6f63 bytes) 2024-09-22T01:15:08.676 INFO:teuthology.orchestra.run.smithi096.stdout:3360 read 0x2068da9 thru 0x2072e75 (0xa0cd bytes) 2024-09-22T01:15:08.677 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:08.677 INFO:teuthology.orchestra.run.smithi096.stdout:3362 writesame 0xe0f574 thru 0xe14386 data_size 0x17(0x4e13 bytes) 2024-09-22T01:15:08.698 INFO:teuthology.orchestra.run.smithi096.stdout:3363 write 0xbe63595 thru 0xbe6b4c5 (0x7f31 bytes) 2024-09-22T01:15:08.718 INFO:teuthology.orchestra.run.smithi096.stdout:3364 trunc from 0xbe6b4c6 to 0x12a337a 2024-09-22T01:15:08.728 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:08.725+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:08.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:15:08.759 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:08.759 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:08.759 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:08.776 INFO:teuthology.orchestra.run.smithi096.stdout:3365 write 0xd00fd61 thru 0xd01fb14 (0xfdb4 bytes) 2024-09-22T01:15:08.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:08.782 INFO:teuthology.orchestra.run.smithi096.stdout:3366 read 0x4c9175a thru 0x4c9cd4b (0xb5f2 bytes) 2024-09-22T01:15:08.782 INFO:teuthology.orchestra.run.smithi096.stdout:3367 punch from 0xc5ec04d to 0xc5fbaa6, (0xfa59 bytes) 2024-09-22T01:15:08.782 INFO:teuthology.orchestra.run.smithi096.stdout:3369 read 0x82e9261 thru 0x82ea39e (0x113e bytes) 2024-09-22T01:15:08.782 INFO:teuthology.orchestra.run.smithi096.stdout:3370 write 0x8f93c10 thru 0x8f95692 (0x1a83 bytes) 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 76, 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:15:07.708672+0000", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:37.510453+0000", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5806080, 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:15:08.789 INFO:teuthology.orchestra.run.smithi096.stdout: "nodeep-scrub", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "noscrub", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:15:08.790 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "luminous", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:15:08.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:15:08.792 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:08.793 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:08.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:08.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:15:08.796 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:08.797 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:08.798 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:08.799 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:15:08.800 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:08.801 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:08.802 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 5, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 5, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:15:08.803 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 5, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 5, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:08.804 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:15:08.805 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "76", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:08.806 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 32, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 76, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:08.807 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:08.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:08.809 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:08.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 57, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.811 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.812 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.813 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.816 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 35, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.817 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.818 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.819 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.820 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.821 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.822 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:15:08.826 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:15:08.827 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:15:08.832 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.833 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.838 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.839 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 55, 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:15:08.844 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.845 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 57, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:15:08.850 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:08.851 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.852 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.853 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.854 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:08.855 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:08.856 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.857 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.858 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.860 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.861 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [ 2024-09-22T01:15:08.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "2.6", 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: 1, 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: 0 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:15:08.863 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [], 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [], 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [], 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:15:08.864 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.865 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:08.866 INFO:teuthology.orchestra.run.smithi096.stdout:3371 trunc from 0xd01fb15 to 0x69c4793 2024-09-22T01:15:08.866 INFO:teuthology.orchestra.run.smithi096.stdout:3372 writesame 0x4c77760 thru 0x4c7cab3 data_size 0x2(0x5354 bytes) 2024-09-22T01:15:08.868 INFO:tasks.thrashosds.thrasher:j is {'epoch': 76, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:15:07.708672+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:37.510453+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 1, 'raw_score_acting': 5, 'raw_score_stable': 5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 76, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [1, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:15:08.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3373 trunc from 0x69c4793 to 0x1e8e858 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3374 punch from 0xab07b4 to 0xabbe94, (0xb6e0 bytes) 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3375 punch from 0x2e3201 to 0x2e85de, (0x53dd bytes) 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3376 read 0x891a37 thru 0x895a89 (0x4053 bytes) 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3378 read 0x15c310f thru 0x15ce638 (0xb52a bytes) 2024-09-22T01:15:08.869 INFO:teuthology.orchestra.run.smithi096.stdout:3379 read 0x173b792 thru 0x174a313 (0xeb82 bytes) 2024-09-22T01:15:08.869 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:08.861+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:08.871 INFO:teuthology.orchestra.run.smithi096.stdout:3380 write 0xdb2907d thru 0xdb2e14e (0x50d2 bytes) 2024-09-22T01:15:08.875 INFO:teuthology.orchestra.run.smithi096.stdout:3381 read 0x72923d7 thru 0x72a1dbf (0xf9e9 bytes) 2024-09-22T01:15:08.891 INFO:teuthology.orchestra.run.smithi096.stdout:3382 trunc from 0xdb2e14f to 0x3f4237b 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6542 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.972290800000003, 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012701107999999999, 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6542 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948069+0000", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948075+0000", 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000003e-06 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948086+0000", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0611000000000001e-05 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948098+0000", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2251e-05 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955277+0000", 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071787439999999999 2024-09-22T01:15:08.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955323+0000", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955513+0000", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019068700000000001 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959026+0000", 2024-09-22T01:15:08.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035129390000000001 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960685+0000", 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016590070000000001 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960726+0000", 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0630999999999998e-05 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960770+0000", 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3609000000000002e-05 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6543 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.972231086999997, 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014741681, 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6543 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948129+0000", 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948133+0000", 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980000000000003e-06 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948148+0000", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4711e-05 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948154+0000", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3849999999999999e-06 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955772+0000", 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076182760000000002 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955821+0000", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8987999999999999e-05 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.956008+0000", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018672799999999999 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959043+0000", 2024-09-22T01:15:08.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030352629999999999 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962789+0000", 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037458420000000001 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962826+0000", 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6545999999999999e-05 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962870+0000", 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4118e-05 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7215 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599136465000001, 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012531801, 2024-09-22T01:15:08.934 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7215 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321239+0000", 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5532999999999998e-05 2024-09-22T01:15:08.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321246+0000", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321255+0000", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4180000000000003e-06 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333660+0000", 2024-09-22T01:15:08.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012404851 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333711+0000", 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1298999999999998e-05 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333755+0000", 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7216 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599095615000003, 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670147999999999, 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7216 2024-09-22T01:15:08.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321266+0000", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321273+0000", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321277+0000", 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7060000000000003e-06 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333792+0000", 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012514208000000001 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333862+0000", 2024-09-22T01:15:08.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0078000000000002e-05 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333934+0000", 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2171999999999999e-05 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7217 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1245184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599075757000001, 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012826168000000001, 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7217 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:15:08.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321286+0000", 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.764e-06 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321292+0000", 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.117e-06 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321297+0000", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7269999999999998e-06 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333973+0000", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012676481 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334042+0000", 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8674e-05 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334110+0000", 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7958000000000002e-05 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7218 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599047356, 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012956225, 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7218 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321313+0000", 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321315+0000", 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-06 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321321+0000", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321326+0000", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.797e-06 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334165+0000", 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01283831 2024-09-22T01:15:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334209+0000", 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4693000000000001e-05 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334268+0000", 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.952 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7219 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599025675999997, 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013092405999999999, 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7219 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321336+0000", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8840000000000001e-06 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321342+0000", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321346+0000", 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-22T01:15:08.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334311+0000", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012964361000000001 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334373+0000", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1791000000000005e-05 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334426+0000", 2024-09-22T01:15:08.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3365000000000002e-05 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7220 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1638400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.599006838999998, 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013199765, 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7220 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321353+0000", 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.22e-07 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321355+0000", 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321362+0000", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321376+0000", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3372e-05 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334459+0000", 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013083539999999999 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334508+0000", 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8754000000000001e-05 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334552+0000", 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7377 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176463834000003, 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013280372, 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7377 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743896+0000", 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743899+0000", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6869999999999999e-06 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743912+0000", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3144e-05 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743919+0000", 2024-09-22T01:15:08.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999994e-06 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747360+0000", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034404280000000001 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747435+0000", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5312e-05 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747724+0000", 2024-09-22T01:15:08.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028894499999999998 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753696+0000", 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059718339999999996 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757027+0000", 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033309030000000001 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757118+0000", 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757176+0000", 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7378 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176432771000002, 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013472072999999999, 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7378 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743927+0000", 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743930+0000", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8559999999999998e-06 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743935+0000", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3199999999999999e-06 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743939+0000", 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3309999999999997e-06 2024-09-22T01:15:08.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748128+0000", 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041883099999999998 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748217+0000", 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9250000000000001e-05 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748581+0000", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036391 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754767+0000", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061866630000000002 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757274+0000", 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025063540000000001 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757344+0000", 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0502999999999999e-05 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757399+0000", 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4242e-05 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7379 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176414557999998, 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013617644999999999, 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7379 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743948+0000", 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743956+0000", 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743961+0000", 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.498e-06 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749603+0000", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056418889999999998 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749676+0000", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2832999999999999e-05 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749988+0000", 2024-09-22T01:15:08.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031244399999999999 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756968+0000", 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069804189999999999 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757467+0000", 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049842199999999999 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757526+0000", 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9268000000000002e-05 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757562+0000", 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6173000000000001e-05 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7380 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.979 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176392796999998, 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013877240000000001, 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7380 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743967+0000", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743970+0000", 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8439999999999998e-06 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743977+0000", 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743981+0000", 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750397+0000", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064165020000000001 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750522+0000", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012448799999999999 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751087+0000", 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000564947 2024-09-22T01:15:08.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757205+0000", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061183619999999996 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757713+0000", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050744799999999997 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757790+0000", 2024-09-22T01:15:08.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7334000000000002e-05 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757844+0000", 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3822000000000001e-05 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7381 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176357363000001, 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014017416, 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7381 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744009+0000", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744018+0000", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0340000000000002e-06 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744023+0000", 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751817+0000", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077938979999999996 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751909+0000", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2436999999999996e-05 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752214+0000", 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030542200000000002 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757581+0000", 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053669049999999999 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757913+0000", 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033169200000000002 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757965+0000", 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1456999999999999e-05 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758019+0000", 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4775000000000003e-05 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7382 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.991 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176329232000001, 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014167732000000001, 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7382 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744033+0000", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744048+0000", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744054+0000", 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8159999999999999e-06 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752687+0000", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086338379999999996 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752828+0000", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014017000000000001 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753320+0000", 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049253600000000002 2024-09-22T01:15:08.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757591+0000", 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042712419999999997 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758086+0000", 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049453699999999997 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758149+0000", 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3261000000000003e-05 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758198+0000", 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7383 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:08.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176298842000001, 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016002188000000001, 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7383 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:08.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744061+0000", 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:08.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744063+0000", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744069+0000", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4829999999999999e-06 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744073+0000", 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753747+0000", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096735599999999995 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753844+0000", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7019000000000001e-05 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754147+0000", 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030294600000000001 2024-09-22T01:15:09.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758259+0000", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004112826 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759947+0000", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001687287 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760009+0000", 2024-09-22T01:15:09.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2697e-05 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760063+0000", 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3202000000000003e-05 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7384 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176280108, 2024-09-22T01:15:09.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016118147999999999, 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7384 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744082+0000", 2024-09-22T01:15:09.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744087+0000", 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744091+0000", 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754832+0000", 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010740774999999999 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754970+0000", 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806199999999999 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.755489+0000", 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051922100000000005 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758312+0000", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028228089999999999 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760075+0000", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001762935 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760136+0000", 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1231999999999999e-05 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760197+0000", 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1033000000000003e-05 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7385 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.176263134000003, 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017662942000000001, 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7385 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744099+0000", 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744113+0000", 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3722e-05 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744120+0000", 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756084+0000", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011963061000000001 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756155+0000", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1326000000000001e-05 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756461+0000", 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030662199999999999 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759930+0000", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034685749999999998 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761621+0000", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001691173 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761703+0000", 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1729000000000003e-05 2024-09-22T01:15:09.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761759+0000", 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6125999999999998e-05 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:19714 3.1d e46/28 3:bd0a6cd9:::rbd_data.11247e2569dd.0000000000000006:head v 46'43, mlcod=41'41)", 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.711946228999999, 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017168761000000001, 2024-09-22T01:15:09.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208415+0000", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9429999999999999e-06 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211325+0000", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029099019999999998 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211356+0000", 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1590000000000001e-05 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211371+0000", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4678000000000001e-05 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211454+0000", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2708999999999997e-05 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211480+0000", 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6264000000000001e-05 2024-09-22T01:15:09.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225527+0000", 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014047074 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225582+0000", 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4601000000000001e-05 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22366 3.c 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.635281526999997, 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022689654, 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22366 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285080+0000", 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.781e-06 2024-09-22T01:15:09.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289468+0000", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043884040000000003 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289523+0000", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5278999999999997e-05 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289551+0000", 2024-09-22T01:15:09.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7574999999999999e-05 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289705+0000", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015408899999999999 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289922+0000", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021745600000000001 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.290362+0000", 2024-09-22T01:15:09.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043983999999999999 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305662+0000", 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015299975 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307559+0000", 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018972259999999999 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307658+0000", 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8330000000000002e-05 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307767+0000", 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010969899999999999 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:29506 3.e e61/58 3:751da8e3:::rbd_data.11242d507bd4.0000000000000006:head v 61'197, mlcod=59'195)", 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 25.05965218, 2024-09-22T01:15:09.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017335876, 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860710+0000", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4830000000000002e-06 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865107+0000", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043974649999999997 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865146+0000", 2024-09-22T01:15:09.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9214000000000001e-05 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865166+0000", 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865300+0000", 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134039 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865338+0000", 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7796000000000002e-05 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878002+0000", 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012663855999999999 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878043+0000", 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1433999999999997e-05 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.030 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.030 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3383 writesame 0xe0446a9 thru 0xe049ee8 data_size 0x20(0x5840 bytes) 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3384 read 0x73a4397 thru 0x73a5f54 (0x1bbe bytes) 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3386 read 0x906f30b thru 0x907bdc0 (0xcab6 bytes) 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3389 punch from 0x11f7f9f to 0x1203147, (0xb1a8 bytes) 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3390 writesame 0x5ec7062 thru 0x5ed484b data_size 0x1d(0xd7ea bytes) 2024-09-22T01:15:09.031 INFO:teuthology.orchestra.run.smithi096.stdout:3391 write 0x31679b6 thru 0x316afa7 (0x35f2 bytes) 2024-09-22T01:15:09.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:09.033 INFO:teuthology.orchestra.run.smithi096.stdout:3393 write 0x4490ee7 thru 0x449ddde (0xcef8 bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3396 write 0x88b052 thru 0x88cb57 (0x1b06 bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3397 write 0x2e1a7e6 thru 0x2e1f204 (0x4a1f bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3398 read 0xcd5c56 thru 0xce04a8 (0xa853 bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3400 read 0x759d952 thru 0x75a76a8 (0x9d57 bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3402 write 0xc2ccdb8 thru 0xc2d48e8 (0x7b31 bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3403 punch from 0xe04972a to 0xe049ee9, (0x7bf bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3405 writesame 0x5749649 thru 0x5758737 data_size 0x1(0xf0ef bytes) 2024-09-22T01:15:09.034 INFO:teuthology.orchestra.run.smithi096.stdout:3407 trunc from 0xe049ee9 to 0x71479c3 2024-09-22T01:15:09.035 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:09.026+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:09.039 INFO:teuthology.orchestra.run.smithi096.stdout:3408 trunc from 0x71479c3 to 0x387e4a0 2024-09-22T01:15:09.075 INFO:teuthology.orchestra.run.smithi096.stdout:3410 write 0xaf53c0a thru 0xaf5ec78 (0xb06f bytes) 2024-09-22T01:15:09.091 INFO:teuthology.orchestra.run.smithi096.stdout:3412 writesame 0xca574df thru 0xca5e61f data_size 0x1(0x7141 bytes) 2024-09-22T01:15:09.092 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.100 INFO:teuthology.orchestra.run.smithi096.stdout:3414 trunc from 0xca5e620 to 0x65613f4 2024-09-22T01:15:09.123 INFO:teuthology.orchestra.run.smithi096.stdout:3415 trunc from 0x65613f4 to 0xe6f46 2024-09-22T01:15:09.132 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:09.129+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:09.182 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:09.182 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:09.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:09.234 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:09.230+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:09.247 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:15:09.268 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-22T01:15:07.717605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":889567582,"num_objects":925,"num_object_clones":249,"num_object_copies":1850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":925,"num_whiteouts":220,"num_read":126790,"num_read_kb":526081,"num_write":9654,"num_write_kb":212115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":43995762,"num_keys_recovered":107,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7742,"ondisk_log_size":7742,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":567448,"kb_used_data":280336,"kb_used_omap":12,"kb_used_meta":286771,"kb_avail":754407272,"statfs":{"total":773094113280,"available":772513046528,"internally_reserved":0,"allocated":287064064,"data_stored":413330264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":293654064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56265719,"num_objects":153,"num_object_clones":24,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":26,"num_read":11933,"num_read_kb":30169,"num_write":1599,"num_write_kb":10150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.176833"},"pg_stats":[{"pgid":"3.1f","version":"72'113","reported_seq":764,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.355656+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:15:03.355656+0000","last_peered":"2024-09-22T01:15:03.355656+0000","last_clean":"2024-09-22T01:15:03.355656+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:15:03.355656+0000","last_undegraded":"2024-09-22T01:15:03.355656+0000","last_fullsized":"2024-09-22T01:15:03.355656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:02:51.419627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":10547260,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":579,"num_read_kb":9486,"num_write":112,"num_write_kb":1589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"72'144","reported_seq":10601,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.351767+0000","last_change":"2024-09-22T01:14:38.655424+0000","last_active":"2024-09-22T01:15:03.351767+0000","last_peered":"2024-09-22T01:15:03.351767+0000","last_clean":"2024-09-22T01:15:03.351767+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:15:03.351767+0000","last_undegraded":"2024-09-22T01:15:03.351767+0000","last_fullsized":"2024-09-22T01:15:03.351767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:31.443722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":32213006,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":10272,"num_read_kb":27749,"num_write":142,"num_write_kb":10170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"72'119","reported_seq":11019,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.497229+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:15:03.497229+0000","last_peered":"2024-09-22T01:15:03.497229+0000","last_clean":"2024-09-22T01:15:03.497229+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:15:03.497229+0000","last_undegraded":"2024-09-22T01:15:03.497229+0000","last_fullsized":"2024-09-22T01:15:03.497229+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:01.444036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":22752656,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":10869,"num_read_kb":15969,"num_write":150,"num_write_kb":8017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"72'149","reported_seq":2306,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:02.668887+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:15:02.668887+0000","last_peered":"2024-09-22T01:15:02.668887+0000","last_clean":"2024-09-22T01:15:02.668887+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:15:02.668887+0000","last_undegraded":"2024-09-22T01:15:02.668887+0000","last_fullsized":"2024-09-22T01:15:02.668887+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:57:03.439435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":32127811,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":4,"num_read":2228,"num_read_kb":17713,"num_write":157,"num_write_kb":8793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"72'130","reported_seq":2268,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:02.922645+0000","last_change":"2024-09-22T01:14:06.510535+0000","last_active":"2024-09-22T01:15:02.922645+0000","last_peered":"2024-09-22T01:15:02.922645+0000","last_clean":"2024-09-22T01:15:02.922645+0000","last_became_active":"2024-09-22T01:14:04.502656+0000","last_became_peered":"2024-09-22T01:14:04.502656+0000","last_unstale":"2024-09-22T01:15:02.922645+0000","last_undegraded":"2024-09-22T01:15:02.922645+0000","last_fullsized":"2024-09-22T01:15:02.922645+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:52.822051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":13535512,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":2316,"num_read_kb":2283,"num_write":116,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"72'415","reported_seq":11527,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.504175+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:15:03.504175+0000","last_peered":"2024-09-22T01:15:03.504175+0000","last_clean":"2024-09-22T01:15:03.504175+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:15:03.504175+0000","last_undegraded":"2024-09-22T01:15:03.504175+0000","last_fullsized":"2024-09-22T01:15:03.504175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:08:30.223728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":40754448,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":12731,"num_read_kb":51621,"num_write":695,"num_write_kb":21361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"72'133","reported_seq":3487,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:02.868628+0000","last_change":"2024-09-22T01:14:06.509219+0000","last_active":"2024-09-22T01:15:02.868628+0000","last_peered":"2024-09-22T01:15:02.868628+0000","last_clean":"2024-09-22T01:15:02.868628+0000","last_became_active":"2024-09-22T01:14:04.511622+0000","last_became_peered":"2024-09-22T01:14:04.511622+0000","last_unstale":"2024-09-22T01:15:02.868628+0000","last_undegraded":"2024-09-22T01:15:02.868628+0000","last_fullsized":"2024-09-22T01:15:02.868628+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:40:10.182817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":35265762,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":3967,"num_read_kb":18833,"num_write":128,"num_write_kb":1957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"72'130","reported_seq":4279,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.127482+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:15:03.127482+0000","last_peered":"2024-09-22T01:15:03.127482+0000","last_clean":"2024-09-22T01:15:03.127482+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:15:03.127482+0000","last_undegraded":"2024-09-22T01:15:03.127482+0000","last_fullsized":"2024-09-22T01:15:03.127482+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:54:52.452963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":39076402,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":4269,"num_read_kb":34684,"num_write":139,"num_write_kb":15345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"72'171","reported_seq":2182,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.466774+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:15:03.466774+0000","last_peered":"2024-09-22T01:15:03.466774+0000","last_clean":"2024-09-22T01:15:03.466774+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:15:03.466774+0000","last_undegraded":"2024-09-22T01:15:03.466774+0000","last_fullsized":"2024-09-22T01:15:03.466774+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:44:11.912757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":32089627,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":2082,"num_read_kb":18860,"num_write":189,"num_write_kb":6180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"72'208","reported_seq":10524,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.351826+0000","last_change":"2024-09-22T01:14:41.577883+0000","last_active":"2024-09-22T01:15:03.351826+0000","last_peered":"2024-09-22T01:15:03.351826+0000","last_clean":"2024-09-22T01:15:03.351826+0000","last_became_active":"2024-09-22T01:14:41.576276+0000","last_became_peered":"2024-09-22T01:14:41.576276+0000","last_unstale":"2024-09-22T01:15:03.351826+0000","last_undegraded":"2024-09-22T01:15:03.351826+0000","last_fullsized":"2024-09-22T01:15:03.351826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:46:18.974762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":22555469,"num_objects":30,"num_object_clones":9,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":7,"num_read":10370,"num_read_kb":18155,"num_write":248,"num_write_kb":7992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4954128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"75'213","reported_seq":880,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-22T01:15:07.409184+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:15:07.409184+0000","last_peered":"2024-09-22T01:15:07.409184+0000","last_clean":"2024-09-22T01:15:07.409184+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:15:07.409184+0000","last_undegraded":"2024-09-22T01:15:07.409184+0000","last_fullsized":"2024-09-22T01:15:07.409184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:53:09.464492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":11806383,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":1054,"num_read_kb":7868,"num_write":283,"num_write_kb":2505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"72'382","reported_seq":1556,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.352056+0000","last_change":"2024-09-22T01:14:41.579663+0000","last_active":"2024-09-22T01:15:03.352056+0000","last_peered":"2024-09-22T01:15:03.352056+0000","last_clean":"2024-09-22T01:15:03.352056+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:15:03.352056+0000","last_undegraded":"2024-09-22T01:15:03.352056+0000","last_fullsized":"2024-09-22T01:15:03.352056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:05:58.174393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":32971173,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":3249,"num_read_kb":7107,"num_write":539,"num_write_kb":5335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.759784+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:15:01.759784+0000","last_peered":"2024-09-22T01:15:01.759784+0000","last_clean":"2024-09-22T01:15:01.759784+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:15:01.759784+0000","last_undegraded":"2024-09-22T01:15:01.759784+0000","last_fullsized":"2024-09-22T01:15:01.759784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"72'249","reported_seq":1984,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.204118+0000","last_change":"2024-09-22T01:14:06.510510+0000","last_active":"2024-09-22T01:15:03.204118+0000","last_peered":"2024-09-22T01:15:03.204118+0000","last_clean":"2024-09-22T01:15:03.204118+0000","last_became_active":"2024-09-22T01:14:04.523389+0000","last_became_peered":"2024-09-22T01:14:04.523389+0000","last_unstale":"2024-09-22T01:15:03.204118+0000","last_undegraded":"2024-09-22T01:15:03.204118+0000","last_fullsized":"2024-09-22T01:15:03.204118+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:31:08.503935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":34338927,"num_objects":31,"num_object_clones":10,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":3247,"num_read_kb":36731,"num_write":246,"num_write_kb":21382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.701660+0000","last_change":"2024-09-22T01:14:17.618532+0000","last_active":"2024-09-22T01:15:01.701660+0000","last_peered":"2024-09-22T01:15:01.701660+0000","last_clean":"2024-09-22T01:15:01.701660+0000","last_became_active":"2024-09-22T01:14:17.618111+0000","last_became_peered":"2024-09-22T01:14:17.618111+0000","last_unstale":"2024-09-22T01:15:01.701660+0000","last_undegraded":"2024-09-22T01:15:01.701660+0000","last_fullsized":"2024-09-22T01:15:01.701660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:51.006648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"72'113","reported_seq":1344,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:02.671708+0000","last_change":"2024-09-22T01:14:06.510012+0000","last_active":"2024-09-22T01:15:02.671708+0000","last_peered":"2024-09-22T01:15:02.671708+0000","last_clean":"2024-09-22T01:15:02.671708+0000","last_became_active":"2024-09-22T01:14:04.529848+0000","last_became_peered":"2024-09-22T01:14:04.529848+0000","last_unstale":"2024-09-22T01:15:02.671708+0000","last_undegraded":"2024-09-22T01:15:02.671708+0000","last_fullsized":"2024-09-22T01:15:02.671708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:23:39.185327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":11340097,"num_objects":25,"num_object_clones":9,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":1711,"num_read_kb":7213,"num_write":110,"num_write_kb":1898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":96,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.759935+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:15:01.759935+0000","last_peered":"2024-09-22T01:15:01.759935+0000","last_clean":"2024-09-22T01:15:01.759935+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:15:01.759935+0000","last_undegraded":"2024-09-22T01:15:01.759935+0000","last_fullsized":"2024-09-22T01:15:01.759935+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"72'113","reported_seq":1252,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.421844+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:15:03.421844+0000","last_peered":"2024-09-22T01:15:03.421844+0000","last_clean":"2024-09-22T01:15:03.421844+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:15:03.421844+0000","last_undegraded":"2024-09-22T01:15:03.421844+0000","last_fullsized":"2024-09-22T01:15:03.421844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:52:03.208409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":11680321,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":7,"num_read":2435,"num_read_kb":7129,"num_write":124,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.686983+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:15:01.686983+0000","last_peered":"2024-09-22T01:15:01.686983+0000","last_clean":"2024-09-22T01:15:01.686983+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:15:01.686983+0000","last_undegraded":"2024-09-22T01:15:01.686983+0000","last_fullsized":"2024-09-22T01:15:01.686983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"72'281","reported_seq":1131,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.351511+0000","last_change":"2024-09-22T01:14:41.570935+0000","last_active":"2024-09-22T01:15:03.351511+0000","last_peered":"2024-09-22T01:15:03.351511+0000","last_clean":"2024-09-22T01:15:03.351511+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:15:03.351511+0000","last_undegraded":"2024-09-22T01:15:03.351511+0000","last_fullsized":"2024-09-22T01:15:03.351511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:23.081253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":22600864,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":1593,"num_read_kb":15207,"num_write":406,"num_write_kb":4734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"75'217","reported_seq":1001,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-22T01:15:07.409183+0000","last_change":"2024-09-22T01:14:38.704568+0000","last_active":"2024-09-22T01:15:07.409183+0000","last_peered":"2024-09-22T01:15:07.409183+0000","last_clean":"2024-09-22T01:15:07.409183+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:15:07.409183+0000","last_undegraded":"2024-09-22T01:15:07.409183+0000","last_fullsized":"2024-09-22T01:15:07.409183+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:43:57.581473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":12740772,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":992,"num_read_kb":7117,"num_write":273,"num_write_kb":5378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":100,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.759529+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:15:01.759529+0000","last_peered":"2024-09-22T01:15:01.759529+0000","last_clean":"2024-09-22T01:15:01.759529+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:15:01.759529+0000","last_undegraded":"2024-09-22T01:15:01.759529+0000","last_fullsized":"2024-09-22T01:15:01.759529+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"72'233","reported_seq":1156,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.508504+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:15:03.508504+0000","last_peered":"2024-09-22T01:15:03.508504+0000","last_clean":"2024-09-22T01:15:03.508504+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:15:03.508504+0000","last_undegraded":"2024-09-22T01:15:03.508504+0000","last_fullsized":"2024-09-22T01:15:03.508504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:15:07.464078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":29069656,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":1599,"num_read_kb":15943,"num_write":293,"num_write_kb":4371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.782390+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:15:01.782390+0000","last_peered":"2024-09-22T01:15:01.782390+0000","last_clean":"2024-09-22T01:15:01.782390+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:15:01.782390+0000","last_undegraded":"2024-09-22T01:15:01.782390+0000","last_fullsized":"2024-09-22T01:15:01.782390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"72'239","reported_seq":1479,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.086623+0000","last_change":"2024-09-22T01:14:38.760146+0000","last_active":"2024-09-22T01:15:03.086623+0000","last_peered":"2024-09-22T01:15:03.086623+0000","last_clean":"2024-09-22T01:15:03.086623+0000","last_became_active":"2024-09-22T01:14:38.548688+0000","last_became_peered":"2024-09-22T01:14:38.548688+0000","last_unstale":"2024-09-22T01:15:03.086623+0000","last_undegraded":"2024-09-22T01:15:03.086623+0000","last_fullsized":"2024-09-22T01:15:03.086623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:32.128586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":19642691,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":2861,"num_read_kb":3730,"num_write":262,"num_write_kb":5867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":3964014,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":96,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.759304+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:15:01.759304+0000","last_peered":"2024-09-22T01:15:01.759304+0000","last_clean":"2024-09-22T01:15:01.759304+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:15:01.759304+0000","last_undegraded":"2024-09-22T01:15:01.759304+0000","last_fullsized":"2024-09-22T01:15:01.759304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"72'223","reported_seq":3987,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.267790+0000","last_change":"2024-09-22T01:14:41.577440+0000","last_active":"2024-09-22T01:15:03.267790+0000","last_peered":"2024-09-22T01:15:03.267790+0000","last_clean":"2024-09-22T01:15:03.267790+0000","last_became_active":"2024-09-22T01:14:41.576983+0000","last_became_peered":"2024-09-22T01:14:41.576983+0000","last_unstale":"2024-09-22T01:15:03.267790+0000","last_undegraded":"2024-09-22T01:15:03.267790+0000","last_fullsized":"2024-09-22T01:15:03.267790+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:30:17.247919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":46560576,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":5418,"num_read_kb":10465,"num_write":225,"num_write_kb":3863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.759589+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:15:01.759589+0000","last_peered":"2024-09-22T01:15:01.759589+0000","last_clean":"2024-09-22T01:15:01.759589+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:15:01.759589+0000","last_undegraded":"2024-09-22T01:15:01.759589+0000","last_fullsized":"2024-09-22T01:15:01.759589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":267,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:01.701439+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:15:01.701439+0000","last_peered":"2024-09-22T01:15:01.701439+0000","last_clean":"2024-09-22T01:15:01.701439+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:15:01.701439+0000","last_undegraded":"2024-09-22T01:15:01.701439+0000","last_fullsized":"2024-09-22T01:15:01.701439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"72'211","reported_seq":1325,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.311292+0000","last_change":"2024-09-22T01:14:06.509555+0000","last_active":"2024-09-22T01:15:03.311292+0000","last_peered":"2024-09-22T01:15:03.311292+0000","last_clean":"2024-09-22T01:15:03.311292+0000","last_became_active":"2024-09-22T01:14:04.523352+0000","last_became_peered":"2024-09-22T01:14:04.523352+0000","last_unstale":"2024-09-22T01:15:03.311292+0000","last_undegraded":"2024-09-22T01:15:03.311292+0000","last_fullsized":"2024-09-22T01:15:03.311292+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:09:02.391092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":54619790,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":1758,"num_read_kb":11506,"num_write":170,"num_write_kb":2472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"72'416","reported_seq":14198,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.257654+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:15:03.257654+0000","last_peered":"2024-09-22T01:15:03.257654+0000","last_clean":"2024-09-22T01:15:03.257654+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:15:03.257654+0000","last_undegraded":"2024-09-22T01:15:03.257654+0000","last_fullsized":"2024-09-22T01:15:03.257654+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:29:16.025122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":85647182,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":8,"num_read":16269,"num_read_kb":67706,"num_write":480,"num_write_kb":19131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"72'218","reported_seq":1789,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.020497+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:15:03.020497+0000","last_peered":"2024-09-22T01:15:03.020497+0000","last_clean":"2024-09-22T01:15:03.020497+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:15:03.020497+0000","last_undegraded":"2024-09-22T01:15:03.020497+0000","last_fullsized":"2024-09-22T01:15:03.020497+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:15:11.800900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":23993334,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":3778,"num_read_kb":7833,"num_write":219,"num_write_kb":3314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"72'281","reported_seq":1835,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.354664+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:15:03.354664+0000","last_peered":"2024-09-22T01:15:03.354664+0000","last_clean":"2024-09-22T01:15:03.354664+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:15:03.354664+0000","last_undegraded":"2024-09-22T01:15:03.354664+0000","last_fullsized":"2024-09-22T01:15:03.354664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:43:37.669870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":37557653,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":6,"num_read":2424,"num_read_kb":10896,"num_write":436,"num_write_kb":7045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"75'323","reported_seq":2410,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-22T01:15:07.480548+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:15:07.480548+0000","last_peered":"2024-09-22T01:15:07.480548+0000","last_clean":"2024-09-22T01:15:07.480548+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:15:07.480548+0000","last_undegraded":"2024-09-22T01:15:07.480548+0000","last_fullsized":"2024-09-22T01:15:07.480548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:43:24.818563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":32327271,"num_objects":59,"num_object_clones":20,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":22,"num_read":2626,"num_read_kb":23290,"num_write":385,"num_write_kb":8082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"72'397","reported_seq":720,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.217609+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:15:03.217609+0000","last_peered":"2024-09-22T01:15:03.217609+0000","last_clean":"2024-09-22T01:15:03.217609+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:15:03.217609+0000","last_undegraded":"2024-09-22T01:15:03.217609+0000","last_fullsized":"2024-09-22T01:15:03.217609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:05.086119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":7398855,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":1772,"num_read_kb":5524,"num_write":619,"num_write_kb":4214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"72'262","reported_seq":1375,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.357364+0000","last_change":"2024-09-22T01:14:41.579603+0000","last_active":"2024-09-22T01:15:03.357364+0000","last_peered":"2024-09-22T01:15:03.357364+0000","last_clean":"2024-09-22T01:15:03.357364+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:15:03.357364+0000","last_undegraded":"2024-09-22T01:15:03.357364+0000","last_fullsized":"2024-09-22T01:15:03.357364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:28:23.318284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":17352958,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":1610,"num_read_kb":7512,"num_write":337,"num_write_kb":6362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"75'179","reported_seq":520,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-22T01:15:07.046086+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:15:07.046086+0000","last_peered":"2024-09-22T01:15:07.046086+0000","last_clean":"2024-09-22T01:15:07.046086+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:15:07.046086+0000","last_undegraded":"2024-09-22T01:15:07.046086+0000","last_fullsized":"2024-09-22T01:15:07.046086+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:31:28.850586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":10341650,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":592,"num_read_kb":5483,"num_write":210,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"72'306","reported_seq":1215,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.497866+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:15:03.497866+0000","last_peered":"2024-09-22T01:15:03.497866+0000","last_clean":"2024-09-22T01:15:03.497866+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:15:03.497866+0000","last_undegraded":"2024-09-22T01:15:03.497866+0000","last_fullsized":"2024-09-22T01:15:03.497866+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:53:11.901844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":8990406,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":2126,"num_read_kb":5535,"num_write":422,"num_write_kb":1990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"72'319","reported_seq":1604,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.355267+0000","last_change":"2024-09-22T01:14:41.582658+0000","last_active":"2024-09-22T01:15:03.355267+0000","last_peered":"2024-09-22T01:15:03.355267+0000","last_clean":"2024-09-22T01:15:03.355267+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:15:03.355267+0000","last_undegraded":"2024-09-22T01:15:03.355267+0000","last_fullsized":"2024-09-22T01:15:03.355267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:18.048448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":17549495,"num_objects":23,"num_object_clones":7,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":3007,"num_read_kb":10892,"num_write":366,"num_write_kb":4363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"72'424","reported_seq":1921,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.466810+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:15:03.466810+0000","last_peered":"2024-09-22T01:15:03.466810+0000","last_clean":"2024-09-22T01:15:03.466810+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:15:03.466810+0000","last_undegraded":"2024-09-22T01:15:03.466810+0000","last_fullsized":"2024-09-22T01:15:03.466810+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:42.742814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":26539814,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":3680,"num_read_kb":10821,"num_write":598,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"72'311","reported_seq":1928,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-22T01:15:03.020690+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:15:03.020690+0000","last_peered":"2024-09-22T01:15:03.020690+0000","last_clean":"2024-09-22T01:15:03.020690+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:15:03.020690+0000","last_undegraded":"2024-09-22T01:15:03.020690+0000","last_fullsized":"2024-09-22T01:15:03.020690+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:23:31.554323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":52989374,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":3190,"num_read_kb":25120,"num_write":414,"num_write_kb":3777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":888977195,"num_objects":921,"num_object_clones":249,"num_object_copies":1842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":921,"num_whiteouts":220,"num_read":126674,"num_read_kb":525981,"num_write":9493,"num_write_kb":209697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":444,"num_bytes_recovered":43995743,"num_keys_recovered":107,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":279842816,"data_stored":409763303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7622,"ondisk_log_size":7622,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63400,"kb_used_data":32216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308440,"statfs":{"total":96636764160,"available":96571842560,"internally_reserved":0,"allocated":32989184,"data_stored":41967149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60700,"kb_used_data":25188,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311140,"statfs":{"total":96636764160,"available":96574607360,"internally_reserved":0,"allocated":25792512,"data_stored":32005898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80860,"kb_used_data":45516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94290980,"statfs":{"total":96636764160,"available":96553963520,"internally_reserved":0,"allocated":46608384,"data_stored":66311314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76260,"kb_used_data":40912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94295580,"statfs":{"total":96636764160,"available":96558673920,"internally_reserved":0,"allocated":41893888,"data_stored":60959106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76500,"kb_used_data":36992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94295340,"statfs":{"total":96636764160,"available":96558428160,"internally_reserved":0,"allocated":37879808,"data_stored":61385973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70048,"kb_used_data":30536,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94301792,"statfs":{"total":96636764160,"available":96565035008,"internally_reserved":0,"allocated":31268864,"data_stored":48908885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73628,"kb_used_data":38284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94298212,"statfs":{"total":96636764160,"available":96561369088,"internally_reserved":0,"allocated":39202816,"data_stored":57113551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66052,"kb_used_data":30692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94305788,"statfs":{"total":96636764160,"available":96569126912,"internally_reserved":0,"allocated":31428608,"data_stored":44678388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":30715904,"data_stored":44435265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37773312,"data_stored":56146687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":30568448,"data_stored":48660446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37191680,"data_stored":61151635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41123840,"data_stored":60648762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45236224,"data_stored":65390018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":25079808,"data_stored":31739837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32153600,"data_stored":41590653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:15:09.270 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [4, 7] 2024-09-22T01:15:09.270 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1', '4', '7'] 2024-09-22T01:15:09.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1 4 7 2024-09-22T01:15:09.271 INFO:teuthology.orchestra.run.smithi096.stdout:3416 write 0x166e632 thru 0x1673e24 (0x57f3 bytes) 2024-09-22T01:15:09.277 INFO:teuthology.orchestra.run.smithi096.stdout:3418 trunc from 0x1673e25 to 0x3ed608c 2024-09-22T01:15:09.285 INFO:teuthology.orchestra.run.smithi096.stdout:3420 writesame 0x2c83713 thru 0x2c89f39 data_size 0xd(0x6827 bytes) 2024-09-22T01:15:09.290 INFO:teuthology.orchestra.run.smithi096.stdout:3422 flatten 2024-09-22T01:15:09.298 INFO:teuthology.orchestra.run.smithi096.stdout:3423 read 0x241b030 thru 0x241fc2e (0x4bff bytes) 2024-09-22T01:15:09.311 INFO:teuthology.orchestra.run.smithi096.stdout:3424 trunc from 0x3ed608c to 0xd6afacf 2024-09-22T01:15:09.324 INFO:teuthology.orchestra.run.smithi096.stdout:3426 trunc from 0xd6afacf to 0xb22daa9 2024-09-22T01:15:09.332 INFO:teuthology.orchestra.run.smithi096.stdout:3427 punch from 0x42db488 to 0x42def0d, (0x3a85 bytes) 2024-09-22T01:15:09.332 INFO:teuthology.orchestra.run.smithi096.stdout:3428 read 0x8adfcf8 thru 0x8ae24a4 (0x27ad bytes) 2024-09-22T01:15:09.332 INFO:teuthology.orchestra.run.smithi096.stdout:3429 writesame 0x91b67de thru 0x91c26cd data_size 0x82(0xbef0 bytes) 2024-09-22T01:15:09.334 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:09.329+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:09.338 INFO:teuthology.orchestra.run.smithi096.stdout:3432 write 0x696343e thru 0x696bb70 (0x8733 bytes) 2024-09-22T01:15:09.341 INFO:teuthology.orchestra.run.smithi096.stdout:3433 punch from 0x7769f7 to 0x776ed1, (0x4da bytes) 2024-09-22T01:15:09.346 INFO:teuthology.orchestra.run.smithi096.stdout:3434 trunc from 0xb22daa9 to 0x7e9b974 2024-09-22T01:15:09.361 INFO:teuthology.orchestra.run.smithi096.stdout:3437 read 0x6bad846 thru 0x6baf6b4 (0x1e6f bytes) 2024-09-22T01:15:09.367 INFO:teuthology.orchestra.run.smithi096.stdout:3438 write 0x80f5706 thru 0x80fe52c (0x8e27 bytes) 2024-09-22T01:15:09.372 INFO:teuthology.orchestra.run.smithi096.stdout:3439 punch from 0x4c8db6e to 0x4c8f781, (0x1c13 bytes) 2024-09-22T01:15:09.375 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:09.375 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:09.375 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:09.377 INFO:teuthology.orchestra.run.smithi096.stdout:3441 trunc from 0x80fe52d to 0x4c26ca9 2024-09-22T01:15:09.386 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:09.393 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:09.404 INFO:teuthology.orchestra.run.smithi096.stdout:3442 writesame 0x5c68fd6 thru 0x5c787b9 data_size 0x5f(0xf7e4 bytes) 2024-09-22T01:15:09.404 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.409 INFO:teuthology.orchestra.run.smithi096.stdout:3444 trunc from 0x5c787ba to 0x2c59f10 2024-09-22T01:15:09.419 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.419 INFO:teuthology.orchestra.run.smithi096.stdout:3449 read 0x5c66a1 thru 0x5c89bc (0x231c bytes) 2024-09-22T01:15:09.428 INFO:teuthology.orchestra.run.smithi096.stdout:3450 write 0x374b0f3 thru 0x37536e3 (0x85f1 bytes) 2024-09-22T01:15:09.435 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:09.430+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:09.446 INFO:teuthology.orchestra.run.smithi096.stdout:3452 write 0x6dd4678 thru 0x6de3273 (0xebfc bytes) 2024-09-22T01:15:09.451 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.451 INFO:teuthology.orchestra.run.smithi096.stdout:3454 read 0x6dc9b1f thru 0x6dd8478 (0xe95a bytes) 2024-09-22T01:15:09.465 INFO:teuthology.orchestra.run.smithi096.stdout:3455 write 0xbdae271 thru 0xbdb0b17 (0x28a7 bytes) 2024-09-22T01:15:09.471 INFO:teuthology.orchestra.run.smithi096.stdout:3456 read 0x5fac672 thru 0x5faca0d (0x39c bytes) 2024-09-22T01:15:09.471 INFO:teuthology.orchestra.run.smithi096.stdout:3460 write 0x369353b thru 0x369ba0f (0x84d5 bytes) 2024-09-22T01:15:09.479 INFO:teuthology.orchestra.run.smithi096.stdout:3462 trunc from 0xbdb0b18 to 0xa7c9f04 2024-09-22T01:15:09.507 INFO:teuthology.orchestra.run.smithi096.stdout:3463 trunc from 0xa7c9f04 to 0x3d9663 2024-09-22T01:15:09.522 INFO:teuthology.orchestra.run.smithi096.stdout:3465 read 0x3b71b thru 0x4a5c6 (0xeeac bytes) 2024-09-22T01:15:09.536 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:09.533+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:09.547 INFO:teuthology.orchestra.run.smithi096.stdout:3466 write 0xb12037c thru 0xb12187e (0x1503 bytes) 2024-09-22T01:15:09.552 INFO:teuthology.orchestra.run.smithi096.stdout:3467 write 0xeac320 thru 0xeb5afe (0x97df bytes) 2024-09-22T01:15:09.557 INFO:teuthology.orchestra.run.smithi096.stdout:3469 read 0x11a6ac0 thru 0x11aad7c (0x42bd bytes) 2024-09-22T01:15:09.557 INFO:teuthology.orchestra.run.smithi096.stdout:3470 punch from 0x18aa4eb to 0x18ac17b, (0x1c90 bytes) 2024-09-22T01:15:09.558 INFO:teuthology.orchestra.run.smithi096.stdout:3471 write 0xb0143a7 thru 0xb0222ce (0xdf28 bytes) 2024-09-22T01:15:09.564 INFO:teuthology.orchestra.run.smithi096.stdout:3472 read 0x369aece thru 0x36a60de (0xb211 bytes) 2024-09-22T01:15:09.564 INFO:teuthology.orchestra.run.smithi096.stdout:3473 read 0x872d49b thru 0x8736cc2 (0x9828 bytes) 2024-09-22T01:15:09.565 INFO:teuthology.orchestra.run.smithi096.stdout:3474 write 0x747d9ae thru 0x74892d4 (0xb927 bytes) 2024-09-22T01:15:09.567 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.581002272000006, 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:15:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:15:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:15:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57902 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4516864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375276843999998, 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055853368, 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57902 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190194+0000", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190196+0000", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190202+0000", 2024-09-22T01:15:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1519999999999997e-06 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190207+0000", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8960000000000002e-06 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245857+0000", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055649864 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245932+0000", 2024-09-22T01:15:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4938999999999995e-05 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246047+0000", 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115036 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375254612999999, 2024-09-22T01:15:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:15:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:15:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375221840999998, 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375200376999999, 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:15:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:15:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:15:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375179434, 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:15:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:15:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375159526000001, 2024-09-22T01:15:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:15:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375139144999999, 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:15:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.375117207999999, 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:15:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.374607627, 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:15:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.64322658, 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.643192276000001, 2024-09-22T01:15:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:15:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:15:09.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.643121632, 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.642969927999999, 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:15:09.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.642754391, 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.642541166000001, 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.635 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:15:09.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:15:09.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.642343042, 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:15:09.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:09.643 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.64213925, 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:15:09.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:15:09.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.641888424999999, 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:15:09.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.641669405, 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:09.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:09.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:15:09.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:09.656 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:09.657 INFO:teuthology.orchestra.run.smithi096.stdout:3475 read 0x1414fd6 thru 0x141e425 (0x9450 bytes) 2024-09-22T01:15:09.657 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:09.658 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:09.658 INFO:teuthology.orchestra.run.smithi096.stdout:3477 trunc from 0xb12187f to 0x23f0f2 2024-09-22T01:15:09.659 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:09.659 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:09.659 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:09.660 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.660 INFO:teuthology.orchestra.run.smithi096.stdout:3479 trunc from 0x23f0f2 to 0x76300f7 2024-09-22T01:15:09.660 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:09.660 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:09.661 INFO:teuthology.orchestra.run.smithi096.stdout:3480 write 0xec1f256 thru 0xec249b0 (0x575b bytes) 2024-09-22T01:15:09.661 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:09.661 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:09.661 INFO:teuthology.orchestra.run.smithi096.stdout:3481 trunc from 0xec249b1 to 0xad962e6 2024-09-22T01:15:09.662 INFO:teuthology.orchestra.run.smithi096.stdout:3482 write 0x22d6cd9 thru 0x22e22d2 (0xb5fa bytes) 2024-09-22T01:15:09.662 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:09.662 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:09.662 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:09.654+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:09.664 INFO:teuthology.orchestra.run.smithi096.stdout:3483 write 0x68a5f6f thru 0x68ad43d (0x74cf bytes) 2024-09-22T01:15:09.670 INFO:teuthology.orchestra.run.smithi096.stdout:3484 write 0xacc1df9 thru 0xacc75f6 (0x57fe bytes) 2024-09-22T01:15:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:3485 write 0xae21df7 thru 0xae2b5b7 (0x97c1 bytes) 2024-09-22T01:15:09.696 INFO:teuthology.orchestra.run.smithi096.stdout:3486 write 0xcc10b02 thru 0xcc209d2 (0xfed1 bytes) 2024-09-22T01:15:09.703 INFO:teuthology.orchestra.run.smithi096.stdout:3488 write 0x3b6e886 thru 0x3b71069 (0x27e4 bytes) 2024-09-22T01:15:09.706 INFO:teuthology.orchestra.run.smithi096.stdout:3489 writesame 0x3d6bb5e thru 0x3d782fe data_size 0x5(0xc7a1 bytes) 2024-09-22T01:15:09.716 INFO:teuthology.orchestra.run.smithi096.stdout:3491 write 0x4b1e9ef thru 0x4b2cc3e (0xe250 bytes) 2024-09-22T01:15:09.718 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:15:09.719 INFO:teuthology.orchestra.run.smithi096.stderr:set 3.1 pg_upmap mapping to [4,7] 2024-09-22T01:15:09.727 INFO:teuthology.orchestra.run.smithi096.stdout:3492 read 0xaf53c0b thru 0xaf58ef2 (0x52e8 bytes) 2024-09-22T01:15:09.741 INFO:teuthology.orchestra.run.smithi096.stdout:3493 trunc from 0xcc209d3 to 0x451044a 2024-09-22T01:15:09.758 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:09.753+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:09.761 INFO:teuthology.orchestra.run.smithi096.stdout:3494 read 0x2df1eb4 thru 0x2dfdd5d (0xbeaa bytes) 2024-09-22T01:15:09.761 INFO:teuthology.orchestra.run.smithi096.stdout:3495 read 0x1a95dd8 thru 0x1aa16ac (0xb8d5 bytes) 2024-09-22T01:15:09.761 INFO:teuthology.orchestra.run.smithi096.stdout:3496 read 0x1c8add7 thru 0x1c92cbe (0x7ee8 bytes) 2024-09-22T01:15:09.761 INFO:teuthology.orchestra.run.smithi096.stdout:3498 punch from 0x13624be to 0x1371af9, (0xf63b bytes) 2024-09-22T01:15:09.761 INFO:teuthology.orchestra.run.smithi096.stdout:3501 trunc from 0x451044a to 0x476ee04 2024-09-22T01:15:09.768 INFO:teuthology.orchestra.run.smithi096.stdout:3502 read 0x3f38ba7 thru 0x3f3c148 (0x35a2 bytes) 2024-09-22T01:15:09.772 INFO:teuthology.orchestra.run.smithi096.stdout:3503 trunc from 0x476ee04 to 0x88e5d6f 2024-09-22T01:15:09.787 INFO:teuthology.orchestra.run.smithi096.stdout:3505 write 0x9f3992d thru 0x9f46b7c (0xd250 bytes) 2024-09-22T01:15:09.792 INFO:teuthology.orchestra.run.smithi096.stdout:3507 read 0x2dc96d thru 0x2e5eae (0x9542 bytes) 2024-09-22T01:15:09.794 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:09.795 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:09.799 INFO:teuthology.orchestra.run.smithi096.stdout:3508 write 0xa228776 thru 0xa232c88 (0xa513 bytes) 2024-09-22T01:15:09.805 INFO:teuthology.orchestra.run.smithi096.stdout:3509 write 0x902b9d2 thru 0x902f157 (0x3786 bytes) 2024-09-22T01:15:09.809 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:09.809 INFO:teuthology.orchestra.run.smithi096.stdout:3512 write 0x49d7090 thru 0x49e3574 (0xc4e5 bytes) 2024-09-22T01:15:09.815 INFO:teuthology.orchestra.run.smithi096.stdout:3513 write 0x5d5b3c1 thru 0x5d5ff8a (0x4bca bytes) 2024-09-22T01:15:09.826 INFO:teuthology.orchestra.run.smithi096.stdout:3514 trunc from 0xa232c89 to 0x5628952 2024-09-22T01:15:09.841 INFO:teuthology.orchestra.run.smithi096.stdout:3515 read 0x27ea3b8 thru 0x27f0bf2 (0x683b bytes) 2024-09-22T01:15:09.841 INFO:teuthology.orchestra.run.smithi096.stdout:3516 read 0x23935dc thru 0x239b34b (0x7d70 bytes) 2024-09-22T01:15:09.841 INFO:teuthology.orchestra.run.smithi096.stdout:3517 write 0x3cd9b56 thru 0x3ce5488 (0xb933 bytes) 2024-09-22T01:15:09.846 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.847 INFO:teuthology.orchestra.run.smithi096.stdout:3521 write 0x1a63531 thru 0x1a6e873 (0xb343 bytes) 2024-09-22T01:15:09.854 INFO:teuthology.orchestra.run.smithi096.stdout:3524 trunc from 0x5628952 to 0x529518f 2024-09-22T01:15:09.859 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:09.853+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:09.860 INFO:teuthology.orchestra.run.smithi096.stdout:3525 punch from 0xb7f75d to 0xb8dd70, (0xe613 bytes) 2024-09-22T01:15:09.871 INFO:teuthology.orchestra.run.smithi096.stdout:3527 trunc from 0x529518f to 0xe727ef1 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:3528 punch from 0xd8ed2ab to 0xd8f8821, (0xb576 bytes) 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:3530 read 0x93fdba6 thru 0x9408f7e (0xb3d9 bytes) 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:3533 read 0x1f92097 thru 0x1f9cf45 (0xaeaf bytes) 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.878 INFO:teuthology.orchestra.run.smithi096.stdout:3536 punch from 0xd19d537 to 0xd1a1509, (0x3fd2 bytes) 2024-09-22T01:15:09.884 INFO:teuthology.orchestra.run.smithi096.stdout:3537 trunc from 0xe727ef1 to 0x97f4621 2024-09-22T01:15:09.885 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:09.886 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:09.891 INFO:teuthology.orchestra.run.smithi096.stdout:3538 read 0x1987e6 thru 0x19e798 (0x5fb3 bytes) 2024-09-22T01:15:09.891 INFO:teuthology.orchestra.run.smithi096.stdout:3539 read 0x847d524 thru 0x848781a (0xa2f7 bytes) 2024-09-22T01:15:09.891 INFO:teuthology.orchestra.run.smithi096.stdout:3540 write 0xfb7c27 thru 0xfbe0e3 (0x64bd bytes) 2024-09-22T01:15:09.899 INFO:teuthology.orchestra.run.smithi096.stdout:3541 trunc from 0x97f4621 to 0xb531574 2024-09-22T01:15:09.907 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:09.907 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:09.909 INFO:teuthology.orchestra.run.smithi096.stdout:3542 punch from 0x90bb463 to 0x90c644e, (0xafeb bytes) 2024-09-22T01:15:09.909 INFO:teuthology.orchestra.run.smithi096.stdout:3544 read 0x35089 thru 0x3ab98 (0x5b10 bytes) 2024-09-22T01:15:09.909 INFO:teuthology.orchestra.run.smithi096.stdout:3545 write 0x285b7a6 thru 0x286793a (0xc195 bytes) 2024-09-22T01:15:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:3547 read 0xa3fc05e thru 0xa40be61 (0xfe04 bytes) 2024-09-22T01:15:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:3550 punch from 0xb486e3a to 0xb4914fc, (0xa6c2 bytes) 2024-09-22T01:15:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:3551 writesame 0x21b1cd6 thru 0x21b9bd2 data_size 0x13(0x7efd bytes) 2024-09-22T01:15:09.920 INFO:teuthology.orchestra.run.smithi096.stdout:3553 read 0x15a2cc0 thru 0x15b1f62 (0xf2a3 bytes) 2024-09-22T01:15:09.920 INFO:teuthology.orchestra.run.smithi096.stdout:3554 writesame 0xa80d10 thru 0xa8faee data_size 0x13(0xeddf bytes) 2024-09-22T01:15:09.927 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.928 INFO:teuthology.orchestra.run.smithi096.stdout:3557 trunc from 0xb531574 to 0xa45546b 2024-09-22T01:15:09.931 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:09.931 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:09.936 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:09.936 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:09.936 INFO:teuthology.orchestra.run.smithi096.stdout:3558 punch from 0x3e3d14f to 0x3e4499a, (0x784b bytes) 2024-09-22T01:15:09.937 INFO:teuthology.orchestra.run.smithi096.stdout:3560 writesame 0xaad621 thru 0xab051d data_size 0x1(0x2efd bytes) 2024-09-22T01:15:09.946 INFO:teuthology.orchestra.run.smithi096.stdout:3562 write 0x72a6b9f thru 0x72af2c1 (0x8723 bytes) 2024-09-22T01:15:09.950 INFO:teuthology.orchestra.run.smithi096.stdout:3563 read 0x14ea48c thru 0x14f8989 (0xe4fe bytes) 2024-09-22T01:15:09.950 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.950 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:09.950 INFO:teuthology.orchestra.run.smithi096.stdout:3567 write 0xa9c553 thru 0xaab294 (0xed42 bytes) 2024-09-22T01:15:09.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:09.954 INFO:teuthology.orchestra.run.smithi096.stdout:3568 trunc from 0xa45546b to 0x9bd8455 2024-09-22T01:15:09.973 INFO:teuthology.orchestra.run.smithi096.stdout:3569 trunc from 0x9bd8455 to 0x737e86 2024-09-22T01:15:09.973 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:09.969+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:10.012 INFO:teuthology.orchestra.run.smithi096.stdout:3570 write 0xd1b7b53 thru 0xd1ba09d (0x254b bytes) 2024-09-22T01:15:10.023 INFO:teuthology.orchestra.run.smithi096.stdout:3571 write 0xe15e3b7 thru 0xe15ec22 (0x86c bytes) 2024-09-22T01:15:10.027 INFO:teuthology.orchestra.run.smithi096.stdout:3573 punch from 0xb6e168f to 0xb6edf2f, (0xc8a0 bytes) 2024-09-22T01:15:10.032 INFO:teuthology.orchestra.run.smithi096.stdout:3574 trunc from 0xe15ec23 to 0x9f60596 2024-09-22T01:15:10.046 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:10.046 INFO:teuthology.orchestra.run.smithi096.stdout:3578 write 0x56e0838 thru 0x56e8b2c (0x82f5 bytes) 2024-09-22T01:15:10.050 INFO:teuthology.orchestra.run.smithi096.stdout:3579 write 0x60d77be thru 0x60df5bb (0x7dfe bytes) 2024-09-22T01:15:10.055 INFO:teuthology.orchestra.run.smithi096.stdout:3580 trunc from 0x9f60596 to 0xaeab0a6 2024-09-22T01:15:10.060 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:10.057+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:10.063 INFO:teuthology.orchestra.run.smithi096.stdout:3583 writesame 0xa1e4716 thru 0xa1f3b11 data_size 0x24(0xf3fc bytes) 2024-09-22T01:15:10.077 INFO:teuthology.orchestra.run.smithi096.stdout:3585 trunc from 0xaeab0a6 to 0x5fb8f3b 2024-09-22T01:15:10.079 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.079 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.092 INFO:teuthology.orchestra.run.smithi096.stdout:3586 read 0x3638ea thru 0x370df7 (0xd50e bytes) 2024-09-22T01:15:10.092 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:10.092 INFO:teuthology.orchestra.run.smithi096.stdout:3588 read 0x5110497 thru 0x5115bd9 (0x5743 bytes) 2024-09-22T01:15:10.093 INFO:teuthology.orchestra.run.smithi096.stdout:3589 read 0x508e9c thru 0x516fec (0xe151 bytes) 2024-09-22T01:15:10.093 INFO:teuthology.orchestra.run.smithi096.stdout:3590 read 0x103bebe thru 0x104907c (0xd1bf bytes) 2024-09-22T01:15:10.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:10.102 INFO:teuthology.orchestra.run.smithi096.stdout:3592 writesame 0x81020f8 thru 0x8109928 data_size 0x1(0x7831 bytes) 2024-09-22T01:15:10.118 INFO:teuthology.orchestra.run.smithi096.stdout:3593 write 0xd5dc74c thru 0xd5e161c (0x4ed1 bytes) 2024-09-22T01:15:10.124 INFO:teuthology.orchestra.run.smithi096.stdout:3595 read 0x6dd0ed1 thru 0x6ddcb69 (0xbc99 bytes) 2024-09-22T01:15:10.124 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:10.124 INFO:teuthology.orchestra.run.smithi096.stdout:3599 punch from 0xb91c566 to 0xb921510, (0x4faa bytes) 2024-09-22T01:15:10.124 INFO:teuthology.orchestra.run.smithi096.stdout:3600 writesame 0x1204c7f thru 0x121430e data_size 0x78(0xf690 bytes) 2024-09-22T01:15:10.131 INFO:teuthology.orchestra.run.smithi096.stdout:3602 read 0x99e1b18 thru 0x99e9991 (0x7e7a bytes) 2024-09-22T01:15:10.140 INFO:teuthology.orchestra.run.smithi096.stdout:3603 trunc from 0xd5e161d to 0x7a95a61 2024-09-22T01:15:10.153 INFO:teuthology.orchestra.run.smithi096.stdout:3604 write 0x67abf63 thru 0x67b6afc (0xab9a bytes) 2024-09-22T01:15:10.157 INFO:teuthology.orchestra.run.smithi096.stdout:3605 read 0x3ec2e77 thru 0x3ecc422 (0x95ac bytes) 2024-09-22T01:15:10.157 INFO:teuthology.orchestra.run.smithi096.stdout:3606 write 0x6750b4b thru 0x67596a1 (0x8b57 bytes) 2024-09-22T01:15:10.161 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:10.158+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:10.179 INFO:teuthology.orchestra.run.smithi096.stdout:3607 write 0xd9bed6d thru 0xd9c9667 (0xa8fb bytes) 2024-09-22T01:15:10.217 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.217 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:10.276 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:10.273+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:10.364 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:10.358+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:10.389 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.389 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.401 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:10.465 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:10.462+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:10.515 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.515 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:10.566 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:10.561+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:10.659 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.659 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.667 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:10.661+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:10.674 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:10.768 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:10.765+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:10.798 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.798 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.812 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:10.869 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:10.866+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:10.935 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:10.936 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:10.950 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:10.994 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:10.990+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:11.072 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:11.069+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:11.120 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.121 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.136 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:11.182 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:11.178+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:11.258 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.258 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.272 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:11.314 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:11.310+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:11.375 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:11.373+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.437 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:11.498 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:11.494+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:11.570 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.571 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.576 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:11.573+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:11.584 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:11.677 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:11.674+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:11.750 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.750 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.753 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:15:11.766 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:11.804 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:11.801+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:11.879 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:11.874+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:11.929 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:11.929 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:11.944 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:11.951 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:11.986 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:11.982+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:12.023 INFO:teuthology.orchestra.run.smithi096.stdout:3608 read 0xa489a00 thru 0xa48d567 (0x3b68 bytes) 2024-09-22T01:15:12.023 INFO:teuthology.orchestra.run.smithi096.stdout:3610 write 0x1d9a5b7 thru 0x1da973e (0xf188 bytes) 2024-09-22T01:15:12.028 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:12.029 INFO:teuthology.orchestra.run.smithi096.stdout:3613 writesame 0x9b351b9 thru 0x9b43d15 data_size 0x1(0xeb5d bytes) 2024-09-22T01:15:12.081 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:12.074+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:12.095 INFO:teuthology.orchestra.run.smithi096.stdout:3614 trunc from 0xd9c9668 to 0xb9ec85e 2024-09-22T01:15:12.108 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:12.108 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:12.117 INFO:teuthology.orchestra.run.smithi096.stdout:3615 write 0x4fbc20d thru 0x4fc7bfd (0xb9f1 bytes) 2024-09-22T01:15:12.123 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:12.124 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:12.124 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:12.125 INFO:teuthology.orchestra.run.smithi096.stdout:3618 punch from 0x35d7226 to 0x35dd985, (0x675f bytes) 2024-09-22T01:15:12.125 INFO:teuthology.orchestra.run.smithi096.stdout:3619 write 0xa798d3f thru 0xa7a836c (0xf62e bytes) 2024-09-22T01:15:12.128 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:12.129 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:12.133 INFO:teuthology.orchestra.run.smithi096.stdout:3620 writesame 0x34b13f5 thru 0x34b2103 data_size 0x1(0xd0f bytes) 2024-09-22T01:15:12.140 INFO:teuthology.orchestra.run.smithi096.stdout:3622 write 0x8276a73 thru 0x827f18a (0x8718 bytes) 2024-09-22T01:15:12.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:12.143 INFO:teuthology.orchestra.run.smithi096.stdout:3625 punch from 0x104d6ec to 0x1055a17, (0x832b bytes) 2024-09-22T01:15:12.143 INFO:teuthology.orchestra.run.smithi096.stdout:3626 write 0xb1d9444 thru 0xb1dd1a8 (0x3d65 bytes) 2024-09-22T01:15:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:12.148 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:12.148 INFO:teuthology.orchestra.run.smithi096.stdout:3627 write 0x60a98d8 thru 0x60aaf1e (0x1647 bytes) 2024-09-22T01:15:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:3628 read 0x88c4130 thru 0x88cb735 (0x7606 bytes) 2024-09-22T01:15:12.164 INFO:teuthology.orchestra.run.smithi096.stdout:3630 writesame 0xdc1e3e9 thru 0xdc29571 data_size 0x1(0xb189 bytes) 2024-09-22T01:15:12.164 INFO:teuthology.orchestra.run.smithi096.stdout:3631 write 0xd06210c thru 0xd071be4 (0xfad9 bytes) 2024-09-22T01:15:12.169 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:12.169 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:12.169 INFO:teuthology.orchestra.run.smithi096.stdout:3632 read 0x9afc623 thru 0x9b08a23 (0xc401 bytes) 2024-09-22T01:15:12.170 INFO:teuthology.orchestra.run.smithi096.stdout:3633 read 0x1e2b7d8 thru 0x1e33aba (0x82e3 bytes) 2024-09-22T01:15:12.170 INFO:teuthology.orchestra.run.smithi096.stdout:3634 punch from 0x66ec335 to 0x66f6e2a, (0xaaf5 bytes) 2024-09-22T01:15:12.177 INFO:teuthology.orchestra.run.smithi096.stdout:3636 write 0xeb4e812 thru 0xeb5afc7 (0xc7b6 bytes) 2024-09-22T01:15:12.181 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:12.181 INFO:teuthology.orchestra.run.smithi096.stdout:3638 write 0x7ea24e1 thru 0x7ea887a (0x639a bytes) 2024-09-22T01:15:12.181 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:12.177+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:12.184 INFO:teuthology.orchestra.run.smithi096.stdout:3639 write 0xe295787 thru 0xe29907f (0x38f9 bytes) 2024-09-22T01:15:12.188 INFO:teuthology.orchestra.run.smithi096.stdout:3640 punch from 0xab841f4 to 0xab89434, (0x5240 bytes) 2024-09-22T01:15:12.188 INFO:teuthology.orchestra.run.smithi096.stdout:3642 writesame 0x3af3b36 thru 0x3af7c22 data_size 0x1(0x40ed bytes) 2024-09-22T01:15:12.194 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:12.199 INFO:teuthology.orchestra.run.smithi096.stdout:3645 punch from 0x46bf879 to 0x46cf2bd, (0xfa44 bytes) 2024-09-22T01:15:12.199 INFO:teuthology.orchestra.run.smithi096.stdout:3646 punch from 0x4bc2f35 to 0x4bc632d, (0x33f8 bytes) 2024-09-22T01:15:12.199 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:12.199 INFO:teuthology.orchestra.run.smithi096.stdout:3649 read 0x9936e2f thru 0x993f476 (0x8648 bytes) 2024-09-22T01:15:12.200 INFO:teuthology.orchestra.run.smithi096.stdout:3651 writesame 0x4fe9bdd thru 0x4ff9bb9 data_size 0x1(0xffdd bytes) 2024-09-22T01:15:12.215 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:12.215 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:12.245 INFO:teuthology.orchestra.run.smithi096.stdout:3652 write 0x9537872 thru 0x953ecde (0x746d bytes) 2024-09-22T01:15:12.246 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:12.246 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:12.253 INFO:teuthology.orchestra.run.smithi096.stdout:3653 write 0x1490d18 thru 0x1491e9e (0x1187 bytes) 2024-09-22T01:15:12.258 INFO:teuthology.orchestra.run.smithi096.stdout:3655 read 0x333aeee thru 0x3346eca (0xbfdd bytes) 2024-09-22T01:15:12.259 INFO:teuthology.orchestra.run.smithi096.stdout:3657 read 0x5c83bfc thru 0x5c91855 (0xdc5a bytes) 2024-09-22T01:15:12.259 INFO:teuthology.orchestra.run.smithi096.stdout:3658 write 0x3b0b8b9 thru 0x3b1b118 (0xf860 bytes) 2024-09-22T01:15:12.265 INFO:teuthology.orchestra.run.smithi096.stdout:3659 write 0x2d95c1a thru 0x2d9f40a (0x97f1 bytes) 2024-09-22T01:15:12.269 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:12.278 INFO:teuthology.orchestra.run.smithi096.stdout:3660 trunc from 0xeb5afc8 to 0x981b58e 2024-09-22T01:15:12.282 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:12.277+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:12.295 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:12.295 INFO:teuthology.orchestra.run.smithi096.stdout:3662 punch from 0x364900a to 0x36542e9, (0xb2df bytes) 2024-09-22T01:15:12.295 INFO:teuthology.orchestra.run.smithi096.stdout:3663 read 0x18c23b5 thru 0x18d209c (0xfce8 bytes) 2024-09-22T01:15:12.295 INFO:teuthology.orchestra.run.smithi096.stdout:3664 write 0x3a6e5c5 thru 0x3a7b9be (0xd3fa bytes) 2024-09-22T01:15:12.298 INFO:teuthology.orchestra.run.smithi096.stdout:3665 write 0x71b7a66 thru 0x71c6b23 (0xf0be bytes) 2024-09-22T01:15:12.304 INFO:teuthology.orchestra.run.smithi096.stdout:3666 write 0x29310e7 thru 0x2938b6b (0x7a85 bytes) 2024-09-22T01:15:12.308 INFO:teuthology.orchestra.run.smithi096.stdout:3667 write 0x7d3c531 thru 0x7d3cdaf (0x87f bytes) 2024-09-22T01:15:12.311 INFO:teuthology.orchestra.run.smithi096.stdout:3669 write 0x2fb616d thru 0x2fbca82 (0x6916 bytes) 2024-09-22T01:15:12.314 INFO:teuthology.orchestra.run.smithi096.stdout:3670 clone 31 order 22 su 32768 sc 10 2024-09-22T01:15:12.320 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:12.321 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:12.321 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:12.321 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:12.321 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:12.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:12.383 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:12.378+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:12.459 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:12.459 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:12.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:12.459 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.23067751, 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:15:12.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:15:12.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:15:12.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:15:12.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:12.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.230602681000001, 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.230564657000002, 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:15:12.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:15:12.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:15:12.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:15:12.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.230515435000001, 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:15:12.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.230479088000003, 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:15:12.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.571606197999998, 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:15:12.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:15:12.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:15:12.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21702 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12582912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.569425451999997, 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012939249999999999, 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.493 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21702 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888663+0000", 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888664+0000", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888671+0000", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6460000000000004e-06 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888690+0000", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9211e-05 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888705+0000", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5277999999999998e-05 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901117+0000", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012411891 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901349+0000", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023167199999999999 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901602+0000", 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025335199999999998 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21719 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.569352967, 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013420118, 2024-09-22T01:15:12.497 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21719 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888735+0000", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888737+0000", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4670000000000001e-06 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888743+0000", 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6959999999999999e-06 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888763+0000", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9261000000000001e-05 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888776+0000", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3728e-05 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901674+0000", 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012897891999999999 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.901902+0000", 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000227569 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902155+0000", 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025350500000000001 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.501 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.569291149999998, 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:15:12.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.166123571, 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.506 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:15:12.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:15:12.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:15:12.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23845 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.489441790000001, 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012821918999999999, 2024-09-22T01:15:12.510 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23845 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968646+0000", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968648+0000", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.237e-06 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968653+0000", 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968671+0000", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7448e-05 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968684+0000", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3689000000000001e-05 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981003+0000", 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012318628999999999 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981219+0000", 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215966 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981468+0000", 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024939 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23847 3.b 3:d2096aa9:::rbd_data.1124acbf3ee1.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.489386164999999, 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013323968, 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23847 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968702+0000", 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968703+0000", 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1570000000000001e-06 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968747+0000", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3931999999999999e-05 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968768+0000", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0635000000000001e-05 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968783+0000", 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5062000000000001e-05 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981547+0000", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.012764467 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.981756+0000", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00020862100000000001 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982026+0000", 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027009400000000002 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23864 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 4980736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.489283544999999, 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013850668, 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23864 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968805+0000", 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968806+0000", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3769999999999999e-06 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968811+0000", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5029999999999997e-06 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968867+0000", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5180999999999999e-05 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968883+0000", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5945000000000001e-05 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982117+0000", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013234133 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982356+0000", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023904800000000001 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982655+0000", 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029948099999999999 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23869 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.489183056999998, 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01438354, 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23869 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968905+0000", 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968906+0000", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3960000000000001e-06 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968914+0000", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0990000000000001e-06 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968933+0000", 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9432e-05 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968947+0000", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3799e-05 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982742+0000", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013795483000000001 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982986+0000", 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024372700000000001 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983289+0000", 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030260399999999999 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.48912249, 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:12.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:15:12.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:15:12.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.489065115000002, 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:15:12.531 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:12.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.488996405000002, 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:15:12.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:15:12.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.488934460999999, 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.488870493999997, 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.544 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:15:12.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:15:12.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:15:12.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.632775354, 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:12.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:15:12.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:12.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:12.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:12.553 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:12.554 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:12.550+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:12.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:12.658 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:12.654+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:12.684 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:12.684 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:12.684 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:12.684 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:12.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:12.753 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:15:12.756 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:12.753+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:12.826 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:12.826 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:12.827 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:12.827 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:12.840 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:12.857 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:12.854+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:12.958 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:12.953+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:12.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6542 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 57.019446561999999, 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012701107999999999, 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6542 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948068+0000", 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948069+0000", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.569999999999999e-07 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948075+0000", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5400000000000003e-06 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948086+0000", 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0611000000000001e-05 2024-09-22T01:15:12.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948098+0000", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2251e-05 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955277+0000", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071787439999999999 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955323+0000", 2024-09-22T01:15:12.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6131999999999998e-05 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955513+0000", 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019068700000000001 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959026+0000", 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035129390000000001 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960685+0000", 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016590070000000001 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960726+0000", 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0630999999999998e-05 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.960770+0000", 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3609000000000002e-05 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:6543 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 57.019386849, 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014741681, 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 6543 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948128+0000", 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948129+0000", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948133+0000", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1980000000000003e-06 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948148+0000", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4711e-05 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.948154+0000", 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3849999999999999e-06 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955772+0000", 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076182760000000002 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.955821+0000", 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8987999999999999e-05 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.956008+0000", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018672799999999999 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.959043+0000", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0030352629999999999 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962789+0000", 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037458420000000001 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962826+0000", 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6545999999999999e-05 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:15.962870+0000", 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4118e-05 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7215 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 983040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646292227000004, 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012531801, 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7215 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321223+0000", 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9599999999999999e-07 2024-09-22T01:15:12.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321239+0000", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5532999999999998e-05 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321246+0000", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7819999999999996e-06 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321255+0000", 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4180000000000003e-06 2024-09-22T01:15:12.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333660+0000", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012404851 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333711+0000", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1298999999999998e-05 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333755+0000", 2024-09-22T01:15:12.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7216 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1114112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646251376999999, 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012670147999999999, 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7216 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321264+0000", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4099999999999999e-07 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321266+0000", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321273+0000", 2024-09-22T01:15:12.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321277+0000", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7060000000000003e-06 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333792+0000", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012514208000000001 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333862+0000", 2024-09-22T01:15:12.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0078000000000002e-05 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333934+0000", 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2171999999999999e-05 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7217 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1245184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646231518999997, 2024-09-22T01:15:12.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012826168000000001, 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7217 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:12.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321284+0000", 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000002e-07 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321286+0000", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.764e-06 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321292+0000", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.117e-06 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321297+0000", 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7269999999999998e-06 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.333973+0000", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012676481 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334042+0000", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8674e-05 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334110+0000", 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7958000000000002e-05 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7218 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1376256~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646203118000003, 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012956225, 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7218 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321312+0000", 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321313+0000", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2399999999999998e-07 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321315+0000", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.154e-06 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321321+0000", 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7229999999999998e-06 2024-09-22T01:15:12.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321326+0000", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.797e-06 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334165+0000", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01283831 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334209+0000", 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4693000000000001e-05 2024-09-22T01:15:12.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334268+0000", 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8924e-05 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7219 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646181437999999, 2024-09-22T01:15:12.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013092405999999999, 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7219 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:12.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321334+0000", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321336+0000", 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8840000000000001e-06 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321342+0000", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0240000000000003e-06 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321346+0000", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334311+0000", 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012964361000000001 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334373+0000", 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1791000000000005e-05 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334426+0000", 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3365000000000002e-05 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7220 3.3 3:c63418ef:::rbd_data.1124636e4501.0000000000000008:7 [read 1638400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.646162601, 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013199765, 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7220 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321352+0000", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321353+0000", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.22e-07 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321355+0000", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8300000000000001e-06 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321362+0000", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.61e-06 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.321376+0000", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3372e-05 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334459+0000", 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013083539999999999 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334508+0000", 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8754000000000001e-05 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.334552+0000", 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4336999999999999e-05 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7377 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223619595999999, 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013280372, 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7377 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743895+0000", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743896+0000", 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1600000000000002e-07 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743899+0000", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6869999999999999e-06 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743912+0000", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3144e-05 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743919+0000", 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8159999999999994e-06 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747360+0000", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034404280000000001 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747435+0000", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5312e-05 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.747724+0000", 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028894499999999998 2024-09-22T01:15:13.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753696+0000", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059718339999999996 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757027+0000", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033309030000000001 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757118+0000", 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1508000000000005e-05 2024-09-22T01:15:13.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757176+0000", 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7379000000000001e-05 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7378 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223588532999997, 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013472072999999999, 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7378 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743926+0000", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743927+0000", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743930+0000", 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8559999999999998e-06 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743935+0000", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3199999999999999e-06 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743939+0000", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3309999999999997e-06 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748128+0000", 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041883099999999998 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748217+0000", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9250000000000001e-05 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.748581+0000", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00036391 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754767+0000", 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061866630000000002 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757274+0000", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025063540000000001 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757344+0000", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0502999999999999e-05 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757399+0000", 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4242e-05 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7379 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.22357032, 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013617644999999999, 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7379 2024-09-22T01:15:13.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743945+0000", 2024-09-22T01:15:13.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743948+0000", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6570000000000001e-06 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743956+0000", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7139999999999999e-06 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743961+0000", 2024-09-22T01:15:13.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.498e-06 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749603+0000", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056418889999999998 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749676+0000", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2832999999999999e-05 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.749988+0000", 2024-09-22T01:15:13.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031244399999999999 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756968+0000", 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0069804189999999999 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757467+0000", 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049842199999999999 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757526+0000", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9268000000000002e-05 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757562+0000", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6173000000000001e-05 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7380 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:13.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223548559000001, 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013877240000000001, 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7380 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743966+0000", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743967+0000", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743970+0000", 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8439999999999998e-06 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743977+0000", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9829999999999999e-06 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.743981+0000", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1799999999999998e-06 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750397+0000", 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0064165020000000001 2024-09-22T01:15:13.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.750522+0000", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012448799999999999 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751087+0000", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000564947 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757205+0000", 2024-09-22T01:15:13.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061183619999999996 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757713+0000", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050744799999999997 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757790+0000", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7334000000000002e-05 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757844+0000", 2024-09-22T01:15:13.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3822000000000001e-05 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7381 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223513124999997, 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014017416, 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7381 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744002+0000", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744009+0000", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2020000000000001e-06 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744018+0000", 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0340000000000002e-06 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744023+0000", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751817+0000", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077938979999999996 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.751909+0000", 2024-09-22T01:15:13.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2436999999999996e-05 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752214+0000", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030542200000000002 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757581+0000", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053669049999999999 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757913+0000", 2024-09-22T01:15:13.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033169200000000002 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757965+0000", 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1456999999999999e-05 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758019+0000", 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4775000000000003e-05 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7382 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223484994000003, 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014167732000000001, 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7382 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744030+0000", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744033+0000", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8710000000000001e-06 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744048+0000", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.438e-05 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744054+0000", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8159999999999999e-06 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752687+0000", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0086338379999999996 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.752828+0000", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014017000000000001 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753320+0000", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049253600000000002 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.757591+0000", 2024-09-22T01:15:13.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042712419999999997 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758086+0000", 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049453699999999997 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758149+0000", 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3261000000000003e-05 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758198+0000", 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8680999999999997e-05 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7383 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223454603999997, 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016002188000000001, 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.040 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7383 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744060+0000", 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744061+0000", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.27e-07 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744063+0000", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6110000000000001e-06 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744069+0000", 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4829999999999999e-06 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744073+0000", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000003e-06 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753747+0000", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0096735599999999995 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.753844+0000", 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7019000000000001e-05 2024-09-22T01:15:13.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754147+0000", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030294600000000001 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758259+0000", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.004112826 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759947+0000", 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001687287 2024-09-22T01:15:13.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760009+0000", 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2697e-05 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760063+0000", 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3202000000000003e-05 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7384 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223435870000003, 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.016118147999999999, 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7384 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744079+0000", 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744082+0000", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6299999999999998e-06 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744087+0000", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0409999999999997e-06 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744091+0000", 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0910000000000003e-06 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754832+0000", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.010740774999999999 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.754970+0000", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013806199999999999 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.755489+0000", 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051922100000000005 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.758312+0000", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028228089999999999 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760075+0000", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001762935 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760136+0000", 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1231999999999999e-05 2024-09-22T01:15:13.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.760197+0000", 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1033000000000003e-05 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:7385 3.15 3:a9ed5b11:::rbd_object_map.11241ea8864d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.223418895999998, 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017662942000000001, 2024-09-22T01:15:13.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7385 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744096+0000", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1899999999999998e-07 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744099+0000", 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7080000000000002e-06 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744113+0000", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3722e-05 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.744120+0000", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5809999999999999e-06 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756084+0000", 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.011963061000000001 2024-09-22T01:15:13.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756155+0000", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1326000000000001e-05 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.756461+0000", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030662199999999999 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.759930+0000", 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0034685749999999998 2024-09-22T01:15:13.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761621+0000", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001691173 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761703+0000", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1729000000000003e-05 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:16.761759+0000", 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6125999999999998e-05 2024-09-22T01:15:13.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:19714 3.1d e46/28 3:bd0a6cd9:::rbd_data.11247e2569dd.0000000000000006:head v 46'43, mlcod=41'41)", 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.759101991000001, 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017168761000000001, 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208413+0000", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.208415+0000", 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9429999999999999e-06 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211325+0000", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029099019999999998 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211356+0000", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1590000000000001e-05 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211371+0000", 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4678000000000001e-05 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211454+0000", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2708999999999997e-05 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.211480+0000", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6264000000000001e-05 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225527+0000", 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014047074 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:29.225582+0000", 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4601000000000001e-05 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22366 3.c 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.682437288999999, 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022689654, 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22366 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:13.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285080+0000", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.781e-06 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289468+0000", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043884040000000003 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289523+0000", 2024-09-22T01:15:13.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5278999999999997e-05 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289551+0000", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7574999999999999e-05 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289705+0000", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015408899999999999 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289922+0000", 2024-09-22T01:15:13.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021745600000000001 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.290362+0000", 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043983999999999999 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305662+0000", 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015299975 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307559+0000", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018972259999999999 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307658+0000", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8330000000000002e-05 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307767+0000", 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010969899999999999 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4388.0:29506 3.e e61/58 3:751da8e3:::rbd_data.11242d507bd4.0000000000000006:head v 61'197, mlcod=59'195)", 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.106807942, 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017335876, 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:13.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860707+0000", 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.860710+0000", 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4830000000000002e-06 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865107+0000", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043974649999999997 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865146+0000", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9214000000000001e-05 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865166+0000", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9589000000000002e-05 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865300+0000", 2024-09-22T01:15:13.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134039 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.865338+0000", 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7796000000000002e-05 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878002+0000", 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012663855999999999 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:43.878043+0000", 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1433999999999997e-05 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.071 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:13.073 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-22T01:15:13.073 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:13.069+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:13.174 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:13.169+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:13.233 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:13.233 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:13.233 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.247 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:13.292 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:13.289+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:13.376 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:13.373+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:13.421 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:13.422 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:13.422 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.436 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:13.480 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:13.477+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:13.578 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:13.574+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:13.604 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:13.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.617331458999999, 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:15:13.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:15:13.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:15:13.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:15:13.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57902 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4516864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411606031000002, 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055853368, 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57902 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190194+0000", 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190196+0000", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190202+0000", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1519999999999997e-06 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190207+0000", 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8960000000000002e-06 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245857+0000", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055649864 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245932+0000", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4938999999999995e-05 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246047+0000", 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115036 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411583799999999, 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:15:13.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:13.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411551028000002, 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411529563999999, 2024-09-22T01:15:13.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411508620999999, 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:15:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:15:13.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411488713000001, 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:13.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:15:13.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:15:13.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411468331999998, 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:15:13.637 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:13.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.411446394999999, 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:15:13.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:15:13.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.410936813999999, 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.679555767, 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:15:13.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.679521463, 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.679450818999999, 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:13.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:15:13.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:15:13.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.679299114999999, 2024-09-22T01:15:13.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:15:13.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:15:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.679083578, 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.678870353000001, 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:15:13.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:15:13.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:15:13.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.678672229, 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.678468436999999, 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:15:13.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:15:13.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.678217612000001, 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:13.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.677998592000002, 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:15:13.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:15:13.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:13.691 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.693 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:13.693 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:13.686+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:13.794 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:13.789+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:13.825 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:13.825 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:13.825 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:13.863 INFO:teuthology.orchestra.run.smithi096.stdout:3671 write 0xea95458 thru 0xea98cbf (0x3868 bytes) 2024-09-22T01:15:13.871 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:13.871 INFO:teuthology.orchestra.run.smithi096.stdout:3674 read 0x12d0f14 thru 0x12dc37b (0xb468 bytes) 2024-09-22T01:15:13.871 INFO:teuthology.orchestra.run.smithi096.stdout:3675 read 0x2d35f1a thru 0x2d3ad58 (0x4e3f bytes) 2024-09-22T01:15:13.871 INFO:teuthology.orchestra.run.smithi096.stdout:3676 write 0x58099cd thru 0x580f5eb (0x5c1f bytes) 2024-09-22T01:15:13.880 INFO:teuthology.orchestra.run.smithi096.stdout:3677 write 0x77899e6 thru 0x778fe65 (0x6480 bytes) 2024-09-22T01:15:13.887 INFO:teuthology.orchestra.run.smithi096.stdout:3678 punch from 0xb2470fc to 0xb249489, (0x238d bytes) 2024-09-22T01:15:13.887 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:13.887 INFO:teuthology.orchestra.run.smithi096.stdout:3680 write 0xd1c51a3 thru 0xd1d18a5 (0xc703 bytes) 2024-09-22T01:15:13.895 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:13.890+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:13.911 INFO:teuthology.orchestra.run.smithi096.stdout:3681 trunc from 0xea98cc0 to 0x5af6f55 2024-09-22T01:15:13.955 INFO:teuthology.orchestra.run.smithi096.stdout:3683 punch from 0x1c0e155 to 0x1c151bd, (0x7068 bytes) 2024-09-22T01:15:13.973 INFO:teuthology.orchestra.run.smithi096.stdout:3684 write 0x9cc8d11 thru 0x9cd0e68 (0x8158 bytes) 2024-09-22T01:15:13.980 INFO:teuthology.orchestra.run.smithi096.stdout:3685 write 0x56f7839 thru 0x5700316 (0x8ade bytes) 2024-09-22T01:15:13.996 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:13.993+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:13.997 INFO:teuthology.orchestra.run.smithi096.stdout:3686 writesame 0xa7c36d8 thru 0xa7c983c data_size 0x1(0x6165 bytes) 2024-09-22T01:15:13.998 INFO:teuthology.orchestra.run.smithi096.stdout:3688 read 0x58314c1 thru 0x583d9ac (0xc4ec bytes) 2024-09-22T01:15:13.998 INFO:teuthology.orchestra.run.smithi096.stdout:3689 read 0x203056a thru 0x2037d5a (0x77f1 bytes) 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout:3691 read 0xa63cd39 thru 0xa64a177 (0xd43f bytes) 2024-09-22T01:15:13.999 INFO:teuthology.orchestra.run.smithi096.stdout:3692 read 0x528e903 thru 0x529e748 (0xfe46 bytes) 2024-09-22T01:15:14.000 INFO:teuthology.orchestra.run.smithi096.stdout:3693 punch from 0x693f30b to 0x6947748, (0x843d bytes) 2024-09-22T01:15:14.000 INFO:teuthology.orchestra.run.smithi096.stdout:3694 punch from 0x2c0f237 to 0x2c1ceac, (0xdc75 bytes) 2024-09-22T01:15:14.000 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:14.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:14.019 INFO:teuthology.orchestra.run.smithi096.stdout:3696 writesame 0xc7274c5 thru 0xc727719 data_size 0x1(0x255 bytes) 2024-09-22T01:15:14.019 INFO:teuthology.orchestra.run.smithi096.stdout:3697 punch from 0x37d1017 to 0x37e0e36, (0xfe1f bytes) 2024-09-22T01:15:14.019 INFO:teuthology.orchestra.run.smithi096.stdout:3698 read 0x4b3cc6f thru 0x4b3d8e7 (0xc79 bytes) 2024-09-22T01:15:14.019 INFO:teuthology.orchestra.run.smithi096.stdout:3699 write 0x8559d8c thru 0x855ad19 (0xf8e bytes) 2024-09-22T01:15:14.022 INFO:teuthology.orchestra.run.smithi096.stdout:3703 write 0xa75e059 thru 0xa76027e (0x2226 bytes) 2024-09-22T01:15:14.030 INFO:teuthology.orchestra.run.smithi096.stdout:3705 read 0xa8109a4 thru 0xa81af9d (0xa5fa bytes) 2024-09-22T01:15:14.031 INFO:teuthology.orchestra.run.smithi096.stdout:3706 write 0x34a9532 thru 0x34b3266 (0x9d35 bytes) 2024-09-22T01:15:14.046 INFO:teuthology.orchestra.run.smithi096.stdout:3707 trunc from 0xc72771a to 0xaa03fe2 2024-09-22T01:15:14.061 INFO:teuthology.orchestra.run.smithi096.stdout:3708 read 0x103a6cd thru 0x1044868 (0xa19c bytes) 2024-09-22T01:15:14.061 INFO:teuthology.orchestra.run.smithi096.stdout:3709 read 0x5a902d7 thru 0x5a9408d (0x3db7 bytes) 2024-09-22T01:15:14.062 INFO:teuthology.orchestra.run.smithi096.stdout:3711 punch from 0x86632dd to 0x8671217, (0xdf3a bytes) 2024-09-22T01:15:14.062 INFO:teuthology.orchestra.run.smithi096.stdout:3712 write 0x2822a53 thru 0x28311de (0xe78c bytes) 2024-09-22T01:15:14.070 INFO:teuthology.orchestra.run.smithi096.stdout:3713 write 0x6b034cf thru 0x6b07b89 (0x46bb bytes) 2024-09-22T01:15:14.074 INFO:teuthology.orchestra.run.smithi096.stdout:3715 read 0x4c46e2c thru 0x4c55a89 (0xec5e bytes) 2024-09-22T01:15:14.074 INFO:teuthology.orchestra.run.smithi096.stdout:3716 write 0x156d11d thru 0x15798a6 (0xc78a bytes) 2024-09-22T01:15:14.096 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:14.093+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:14.103 INFO:teuthology.orchestra.run.smithi096.stdout:3717 write 0xd5137c6 thru 0xd516fd4 (0x380f bytes) 2024-09-22T01:15:14.110 INFO:teuthology.orchestra.run.smithi096.stdout:3718 read 0x905271b thru 0x905b304 (0x8bea bytes) 2024-09-22T01:15:14.111 INFO:teuthology.orchestra.run.smithi096.stdout:3719 write 0x433f92e thru 0x4347161 (0x7834 bytes) 2024-09-22T01:15:14.124 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:14.136 INFO:teuthology.orchestra.run.smithi096.stdout:3722 write 0xdee29f3 thru 0xdee7b15 (0x5123 bytes) 2024-09-22T01:15:14.145 INFO:teuthology.orchestra.run.smithi096.stdout:3724 read 0x3408df8 thru 0x341662e (0xd837 bytes) 2024-09-22T01:15:14.154 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.257307466, 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:15:14.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:14.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.257143110000001, 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:15:14.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:15:14.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252451334, 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252413780000001, 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252381618000001, 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:15:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:15:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:15:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252359281, 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252337920999999, 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.252316731000001, 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:15:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:15:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248411237999999, 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248389057000001, 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248367778999999, 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:14.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:15:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248345432000001, 2024-09-22T01:15:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:15:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:15:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248323756000001, 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:15:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:15:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.248302334000002, 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:15:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.241663589000002, 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.241446841999998, 2024-09-22T01:15:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:15:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:15:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.241064215000002, 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.240658674999999, 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.237442214000001, 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.200656818999999, 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:14.241 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:14.241 INFO:teuthology.orchestra.run.smithi096.stdout:3725 trunc from 0xdee7b16 to 0x3ef0304 2024-09-22T01:15:14.242 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:14.243 INFO:teuthology.orchestra.run.smithi096.stdout:3726 trunc from 0x3ef0304 to 0xdf0b5e4 2024-09-22T01:15:14.243 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:14.237+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:14.244 INFO:teuthology.orchestra.run.smithi096.stdout:3727 writesame 0x3a47e92 thru 0x3a4c7d7 data_size 0x53(0x4946 bytes) 2024-09-22T01:15:14.249 INFO:teuthology.orchestra.run.smithi096.stdout:3729 writesame 0xb4c44ba thru 0xb4d0dcc data_size 0x47(0xc913 bytes) 2024-09-22T01:15:14.260 INFO:teuthology.orchestra.run.smithi096.stdout:3730 writesame 0x8c86539 thru 0x8c91d8a data_size 0x1(0xb852 bytes) 2024-09-22T01:15:14.297 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:14.300 INFO:teuthology.orchestra.run.smithi096.stdout:3731 writesame 0xce68117 thru 0xce72266 data_size 0x3a(0xa150 bytes) 2024-09-22T01:15:14.330 INFO:teuthology.orchestra.run.smithi096.stdout:3732 trunc from 0xdf0b5e4 to 0x1fff100 2024-09-22T01:15:14.343 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:14.338+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:14.381 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:14.381 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:14.381 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:14.382 INFO:teuthology.orchestra.run.smithi096.stdout:3733 write 0x1eddcd3 thru 0x1ee87e6 (0xab14 bytes) 2024-09-22T01:15:14.385 INFO:teuthology.orchestra.run.smithi096.stdout:3734 read 0x1ee8db7 thru 0x1eec4a8 (0x36f2 bytes) 2024-09-22T01:15:14.403 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:14.409 INFO:teuthology.orchestra.run.smithi096.stdout:3735 write 0x8cc9305 thru 0x8cce183 (0x4e7f bytes) 2024-09-22T01:15:14.415 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:14.429 INFO:teuthology.orchestra.run.smithi096.stdout:3737 write 0x913db1b thru 0x914c50f (0xe9f5 bytes) 2024-09-22T01:15:14.439 INFO:teuthology.orchestra.run.smithi096.stdout:3740 clone 32 order 19 su 131072 sc 9 2024-09-22T01:15:14.443 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:14.437+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:14.469 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:14.469 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:14.486 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:14.486 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:14.504 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:14.504 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:14.520 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:14.520 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:14.527 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:14.527 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:14.541 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:14.541 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:14.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:14.545 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:14.538+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:14.561 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:14.562 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:14.582 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:14.582 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:14.603 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:14.604 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:14.645 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:14.638+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:14.677 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:14.677 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:14.691 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:14.746 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:14.741+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:14.759 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:14.759 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:14.760 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:15:14.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:15:14.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:15:14.827 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:14.827 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:14.842 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:14.852 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:14.849+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:14.878 INFO:teuthology.orchestra.run.smithi096.stdout:flattening image image_client.0-clone31 2024-09-22T01:15:14.948 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:14.945+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:14.967 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:14.967 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:14.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:15.011 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-22T01:15:15.049 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:15.042+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:15.118 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:15.118 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:15.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:15.142 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 82, 2024-09-22T01:15:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:15:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:15:14.772394+0000", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:37.510453+0000", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5799936, 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:15:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "luminous", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:15:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:15:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:15:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 5, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 5, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 5, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 5, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:15:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:15:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "82", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 34, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 82, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:15:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:15:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:15:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 1, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 1, 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 1 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:15:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 57, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:15:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:15:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 35, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:15:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 79, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 80, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:15:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 55, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:15:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:15:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:15:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:15:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 57, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:15:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 24, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:15:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:15:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:15:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:15:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:15:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [ 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "2.6", 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: 1, 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: 0 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.1", 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: 4, 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: 7 2024-09-22T01:15:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:15:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [], 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [], 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [], 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:15:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:15:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:15.215 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:15.216 INFO:tasks.thrashosds.thrasher:j is {'epoch': 82, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:15:14.772394+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:37.510453+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 1, 'raw_score_acting': 5, 'raw_score_stable': 5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.6', 'osds': [1, 0]}, {'pgid': '3.1', 'osds': [4, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:15:15.217 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.6 2024-09-22T01:15:15.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.6 2024-09-22T01:15:15.217 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:15.210+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:15.317 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:15.310+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:15.327 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:15.327 INFO:teuthology.orchestra.run.smithi096.stdout:3742 write 0x5ef69c6 thru 0x5efcb2c (0x6167 bytes) 2024-09-22T01:15:15.339 INFO:teuthology.orchestra.run.smithi096.stdout:3743 write 0x8562e9e thru 0x8564687 (0x17ea bytes) 2024-09-22T01:15:15.341 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:15.341 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:15.347 INFO:teuthology.orchestra.run.smithi096.stdout:3744 write 0x29a3518 thru 0x29ab8aa (0x8393 bytes) 2024-09-22T01:15:15.353 INFO:teuthology.orchestra.run.smithi096.stdout:3745 write 0x1b5503e thru 0x1b60c5e (0xbc21 bytes) 2024-09-22T01:15:15.356 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:15.374 INFO:teuthology.orchestra.run.smithi096.stdout:3747 trunc from 0x914c510 to 0xd9327fd 2024-09-22T01:15:15.383 INFO:teuthology.orchestra.run.smithi096.stdout:3751 punch from 0xabcf3ba to 0xabd3e57, (0x4a9d bytes) 2024-09-22T01:15:15.383 INFO:teuthology.orchestra.run.smithi096.stdout:3752 writesame 0x3f06b4f thru 0x3f0a59c data_size 0x2(0x3a4e bytes) 2024-09-22T01:15:15.391 INFO:teuthology.orchestra.run.smithi096.stdout:3753 punch from 0x876d9a to 0x877985, (0xbeb bytes) 2024-09-22T01:15:15.391 INFO:teuthology.orchestra.run.smithi096.stdout:3754 write 0x5bfbe99 thru 0x5c00336 (0x449e bytes) 2024-09-22T01:15:15.416 INFO:teuthology.orchestra.run.smithi096.stdout:3755 trunc from 0xd9327fd to 0xc456f4 2024-09-22T01:15:15.417 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:15.413+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:15.484 INFO:teuthology.orchestra.run.smithi096.stdout:3756 write 0xe13e73c thru 0xe14a538 (0xbdfd bytes) 2024-09-22T01:15:15.503 INFO:teuthology.orchestra.run.smithi096.stdout:3757 trunc from 0xe14a539 to 0x620b91e 2024-09-22T01:15:15.519 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:15.514+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:15.522 INFO:teuthology.orchestra.run.smithi096.stdout:3758 writesame 0x2607812 thru 0x2613bf6 data_size 0x2f(0xc3e5 bytes) 2024-09-22T01:15:15.528 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:15.528 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:15.529 INFO:teuthology.orchestra.run.smithi096.stdout:3759 punch from 0x5410bf9 to 0x5415724, (0x4b2b bytes) 2024-09-22T01:15:15.529 INFO:teuthology.orchestra.run.smithi096.stdout:3760 punch from 0x3310fa2 to 0x3314217, (0x3275 bytes) 2024-09-22T01:15:15.536 INFO:teuthology.orchestra.run.smithi096.stdout:3761 trunc from 0x620b91e to 0x1488d28 2024-09-22T01:15:15.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:15.566 INFO:teuthology.orchestra.run.smithi096.stdout:3762 write 0x203ef25 thru 0x204748b (0x8567 bytes) 2024-09-22T01:15:15.585 INFO:teuthology.orchestra.run.smithi096.stdout:3764 write 0x2872d8a thru 0x287ba7c (0x8cf3 bytes) 2024-09-22T01:15:15.591 INFO:teuthology.orchestra.run.smithi096.stdout:3767 read 0x1213819 thru 0x121c599 (0x8d81 bytes) 2024-09-22T01:15:15.611 INFO:teuthology.orchestra.run.smithi096.stdout:3769 writesame 0x7337c62 thru 0x734464a data_size 0x25(0xc9e9 bytes) 2024-09-22T01:15:15.611 INFO:teuthology.orchestra.run.smithi096.stdout:3770 read 0xeeabd2 thru 0xef2f2f (0x835e bytes) 2024-09-22T01:15:15.612 INFO:teuthology.orchestra.run.smithi096.stdout:3771 read 0x33acb6 thru 0x347b4a (0xce95 bytes) 2024-09-22T01:15:15.612 INFO:teuthology.orchestra.run.smithi096.stdout:3772 write 0x5930e1b thru 0x5932703 (0x18e9 bytes) 2024-09-22T01:15:15.619 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:15.613+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:15.620 INFO:teuthology.orchestra.run.smithi096.stdout:3773 trunc from 0x734464b to 0x879223a 2024-09-22T01:15:15.637 INFO:teuthology.orchestra.run.smithi096.stdout:3775 trunc from 0x879223a to 0xcefbc1a 2024-09-22T01:15:15.647 INFO:teuthology.orchestra.run.smithi096.stdout:3776 clone 33 order 21 su 131072 sc 4 2024-09-22T01:15:15.707 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:15.708 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:15.720 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:15.717+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:15.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:15.794 INFO:teuthology.orchestra.run.smithi096.stderr:clear 2.6 pg_upmap mapping 2024-09-22T01:15:15.803 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:15:15.820 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:15.814+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:15.848 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:15.849 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:15.862 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:15.910 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone32 from 0xcefbc1a (overlap 0xc456f4) to 0x4e892e 2024-09-22T01:15:15.921 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:15.917+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:16.020 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-22T01:15:16.022 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:16.018+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:16.039 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.039 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.053 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:16.123 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:16.117+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:16.222 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.222 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.224 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:16.217+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:16.236 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:16.324 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:16.321+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:16.355 INFO:teuthology.orchestra.run.smithi096.stdout:3777 writesame 0x3ac863c thru 0x3ad1587 data_size 0x12(0x8f4c bytes) 2024-09-22T01:15:16.359 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.359 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.371 INFO:teuthology.orchestra.run.smithi096.stdout:3778 write 0x2fcf849 thru 0x2fd8819 (0x8fd1 bytes) 2024-09-22T01:15:16.373 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:16.378 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:16.378 INFO:teuthology.orchestra.run.smithi096.stdout:3780 write 0x27a7cbd thru 0x27b159e (0x98e2 bytes) 2024-09-22T01:15:16.385 INFO:teuthology.orchestra.run.smithi096.stdout:3781 read 0x2983e91 thru 0x29848f3 (0xa63 bytes) 2024-09-22T01:15:16.385 INFO:teuthology.orchestra.run.smithi096.stdout:3784 write 0x187317e thru 0x1874b31 (0x19b4 bytes) 2024-09-22T01:15:16.392 INFO:teuthology.orchestra.run.smithi096.stdout:3786 write 0xaf765a2 thru 0xaf85a1c (0xf47b bytes) 2024-09-22T01:15:16.400 INFO:teuthology.orchestra.run.smithi096.stdout:3787 read 0x23d9e5c thru 0x23e92bd (0xf462 bytes) 2024-09-22T01:15:16.416 INFO:teuthology.orchestra.run.smithi096.stdout:3788 write 0xee252b5 thru 0xee286d3 (0x341f bytes) 2024-09-22T01:15:16.422 INFO:teuthology.orchestra.run.smithi096.stdout:3789 punch from 0x4aeb233 to 0x4af87a8, (0xd575 bytes) 2024-09-22T01:15:16.422 INFO:teuthology.orchestra.run.smithi096.stdout:3790 write 0x945e00f thru 0x946dfce (0xffc0 bytes) 2024-09-22T01:15:16.425 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:16.421+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:16.432 INFO:teuthology.orchestra.run.smithi096.stdout:3791 read 0x8152a78 thru 0x815f123 (0xc6ac bytes) 2024-09-22T01:15:16.432 INFO:teuthology.orchestra.run.smithi096.stdout:3792 read 0xbf4301b thru 0xbf45ffe (0x2fe4 bytes) 2024-09-22T01:15:16.432 INFO:teuthology.orchestra.run.smithi096.stdout:3793 write 0xcf0d59b thru 0xcf118d8 (0x433e bytes) 2024-09-22T01:15:16.439 INFO:teuthology.orchestra.run.smithi096.stdout:3794 read 0x6038fb9 thru 0x604851d (0xf565 bytes) 2024-09-22T01:15:16.439 INFO:teuthology.orchestra.run.smithi096.stdout:3795 read 0x8fdffd4 thru 0x8fedf1b (0xdf48 bytes) 2024-09-22T01:15:16.439 INFO:teuthology.orchestra.run.smithi096.stdout:3796 punch from 0x3b92a63 to 0x3ba0748, (0xdce5 bytes) 2024-09-22T01:15:16.439 INFO:teuthology.orchestra.run.smithi096.stdout:3798 read 0x34a30c8 thru 0x34b0a9c (0xd9d5 bytes) 2024-09-22T01:15:16.440 INFO:teuthology.orchestra.run.smithi096.stdout:3799 writesame 0xa471f1a thru 0xa47df2a data_size 0x1(0xc011 bytes) 2024-09-22T01:15:16.495 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.495 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:16.521 INFO:teuthology.orchestra.run.smithi096.stdout:3800 read 0x2855490 thru 0x285bf2e (0x6a9f bytes) 2024-09-22T01:15:16.527 INFO:teuthology.orchestra.run.smithi096.stdout:3801 write 0xaa9e200 thru 0xaaac11a (0xdf1b bytes) 2024-09-22T01:15:16.534 INFO:teuthology.orchestra.run.smithi096.stdout:3803 punch from 0x3ee1d83 to 0x3ee4255, (0x24d2 bytes) 2024-09-22T01:15:16.534 INFO:teuthology.orchestra.run.smithi096.stdout:3804 write 0x1585613 thru 0x158a968 (0x5356 bytes) 2024-09-22T01:15:16.542 INFO:teuthology.orchestra.run.smithi096.stdout:3805 writesame 0xe010c77 thru 0xe013319 data_size 0x3f(0x26a3 bytes) 2024-09-22T01:15:16.551 INFO:teuthology.orchestra.run.smithi096.stdout:3807 write 0xfa74b1 thru 0xfb5a52 (0xe5a2 bytes) 2024-09-22T01:15:16.553 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:16.550+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:16.561 INFO:teuthology.orchestra.run.smithi096.stdout:3810 write 0xb186906 thru 0xb194cc5 (0xe3c0 bytes) 2024-09-22T01:15:16.570 INFO:teuthology.orchestra.run.smithi096.stdout:3811 write 0xd9af211 thru 0xd9bd56c (0xe35c bytes) 2024-09-22T01:15:16.577 INFO:teuthology.orchestra.run.smithi096.stdout:3814 write 0x3d9c47a thru 0x3da17d6 (0x535d bytes) 2024-09-22T01:15:16.588 INFO:teuthology.orchestra.run.smithi096.stdout:3815 write 0x4e28633 thru 0x4e36db9 (0xe787 bytes) 2024-09-22T01:15:16.596 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:16.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:16.631 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.631 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.631 INFO:teuthology.orchestra.run.smithi096.stdout:3817 trunc from 0xee286d4 to 0x1b07243 2024-09-22T01:15:16.632 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:16.629+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:16.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:16.698 INFO:teuthology.orchestra.run.smithi096.stdout:3818 writesame 0xbdec852 thru 0xbdfc1be data_size 0x1(0xf96d bytes) 2024-09-22T01:15:16.698 INFO:teuthology.orchestra.run.smithi096.stdout:3819 punch from 0x8ccdddc to 0x8cd8fd5, (0xb1f9 bytes) 2024-09-22T01:15:16.702 INFO:teuthology.orchestra.run.smithi096.stdout:3820 trunc from 0xbdfc1bf to 0x8d9abb0 2024-09-22T01:15:16.710 INFO:teuthology.orchestra.run.smithi096.stdout:3821 write 0x197e231 thru 0x197f975 (0x1745 bytes) 2024-09-22T01:15:16.718 INFO:teuthology.orchestra.run.smithi096.stdout:3822 trunc from 0x8d9abb0 to 0x4c1eea8 2024-09-22T01:15:16.727 INFO:teuthology.orchestra.run.smithi096.stdout:3823 read 0x24fea9c thru 0x2500be1 (0x2146 bytes) 2024-09-22T01:15:16.727 INFO:teuthology.orchestra.run.smithi096.stdout:3824 punch from 0xef6ad0 to 0xef9f1e, (0x344e bytes) 2024-09-22T01:15:16.727 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:16.721+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:16.742 INFO:teuthology.orchestra.run.smithi096.stdout:3825 write 0x76d51ce thru 0x76e1bad (0xc9e0 bytes) 2024-09-22T01:15:16.749 INFO:teuthology.orchestra.run.smithi096.stdout:3826 read 0x2412acb thru 0x241de91 (0xb3c7 bytes) 2024-09-22T01:15:16.749 INFO:teuthology.orchestra.run.smithi096.stdout:3829 writesame 0x5f29007 thru 0x5f367a2 data_size 0x2(0xd79c bytes) 2024-09-22T01:15:16.772 INFO:teuthology.orchestra.run.smithi096.stdout:3830 read 0x3477fa2 thru 0x347c7dd (0x483c bytes) 2024-09-22T01:15:16.773 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:16.782 INFO:teuthology.orchestra.run.smithi096.stdout:3832 write 0x8e59db2 thru 0x8e65107 (0xb356 bytes) 2024-09-22T01:15:16.785 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:16.786 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:16.790 INFO:teuthology.orchestra.run.smithi096.stdout:3833 write 0x1aac84e thru 0x1ab893c (0xc0ef bytes) 2024-09-22T01:15:16.794 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:16.794 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:16.794 INFO:teuthology.orchestra.run.smithi096.stdout:3834 punch from 0xc23078 to 0xc25688, (0x2610 bytes) 2024-09-22T01:15:16.794 INFO:teuthology.orchestra.run.smithi096.stdout:3835 read 0x96739 thru 0xa3fdf (0xd8a7 bytes) 2024-09-22T01:15:16.795 INFO:teuthology.orchestra.run.smithi096.stdout:3836 write 0xa8218e thru 0xa8746a (0x52dd bytes) 2024-09-22T01:15:16.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:16.805 INFO:teuthology.orchestra.run.smithi096.stdout:3837 read 0x614ea23 thru 0x615c59a (0xdb78 bytes) 2024-09-22T01:15:16.815 INFO:teuthology.orchestra.run.smithi096.stdout:3838 write 0xa16903a thru 0xa16bc4c (0x2c13 bytes) 2024-09-22T01:15:16.819 INFO:teuthology.orchestra.run.smithi096.stdout:3839 read 0x415f808 thru 0x4169502 (0x9cfb bytes) 2024-09-22T01:15:16.819 INFO:teuthology.orchestra.run.smithi096.stdout:3840 writesame 0x6ecd76f thru 0x6ed74a5 data_size 0x1(0x9d37 bytes) 2024-09-22T01:15:16.828 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:16.825+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:16.856 INFO:teuthology.orchestra.run.smithi096.stdout:3841 writesame 0xaf4b7fa thru 0xaf59acb data_size 0x1(0xe2d2 bytes) 2024-09-22T01:15:16.857 INFO:teuthology.orchestra.run.smithi096.stdout:3842 punch from 0x176e3fb to 0x177c550, (0xe155 bytes) 2024-09-22T01:15:16.857 INFO:teuthology.orchestra.run.smithi096.stdout:3843 read 0x1bc553c thru 0x1bcd77a (0x823f bytes) 2024-09-22T01:15:16.857 INFO:teuthology.orchestra.run.smithi096.stdout:3844 punch from 0x7344a04 to 0x7349d3b, (0x5337 bytes) 2024-09-22T01:15:16.864 INFO:teuthology.orchestra.run.smithi096.stdout:3846 trunc from 0xaf59acc to 0x56c476b 2024-09-22T01:15:16.899 INFO:teuthology.orchestra.run.smithi096.stdout:3848 write 0x64f5107 thru 0x64fde62 (0x8d5c bytes) 2024-09-22T01:15:16.916 INFO:teuthology.orchestra.run.smithi096.stdout:3849 write 0x7c8926f thru 0x7c95025 (0xbdb7 bytes) 2024-09-22T01:15:16.921 INFO:teuthology.orchestra.run.smithi096.stdout:3850 read 0x2e2f996 thru 0x2e38529 (0x8b94 bytes) 2024-09-22T01:15:16.921 INFO:teuthology.orchestra.run.smithi096.stdout:3851 read 0x71c9051 thru 0x71d5a19 (0xc9c9 bytes) 2024-09-22T01:15:16.921 INFO:teuthology.orchestra.run.smithi096.stdout:3852 read 0x150eb0c thru 0x1513536 (0x4a2b bytes) 2024-09-22T01:15:16.921 INFO:teuthology.orchestra.run.smithi096.stdout:3853 writesame 0x3ae4786 thru 0x3aefc2b data_size 0x13(0xb4a6 bytes) 2024-09-22T01:15:16.927 INFO:teuthology.orchestra.run.smithi096.stdout:3854 read 0x6dd21a0 thru 0x6dd49dd (0x283e bytes) 2024-09-22T01:15:16.929 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:16.926+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:16.942 INFO:teuthology.orchestra.run.smithi096.stdout:3855 write 0x993eea0 thru 0x99457fb (0x695c bytes) 2024-09-22T01:15:16.943 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:16.943 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:16.943 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:16.950 INFO:teuthology.orchestra.run.smithi096.stdout:3857 write 0x84552aa thru 0x845e85b (0x95b2 bytes) 2024-09-22T01:15:16.955 INFO:teuthology.orchestra.run.smithi096.stdout:3858 writesame 0x6a1d362 thru 0x6a1d4ff data_size 0x2(0x19e bytes) 2024-09-22T01:15:16.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:16.970 INFO:teuthology.orchestra.run.smithi096.stdout:3859 trunc from 0x99457fc to 0xcd0e5ed 2024-09-22T01:15:16.978 INFO:teuthology.orchestra.run.smithi096.stdout:3863 punch from 0x531d5aa to 0x532cbdc, (0xf632 bytes) 2024-09-22T01:15:16.978 INFO:teuthology.orchestra.run.smithi096.stdout:3866 write 0x539fa76 thru 0x53a4fb5 (0x5540 bytes) 2024-09-22T01:15:16.988 INFO:teuthology.orchestra.run.smithi096.stdout:3867 trunc from 0xcd0e5ed to 0x9f957c9 2024-09-22T01:15:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:3869 trunc from 0x9f957c9 to 0xb0a6250 2024-09-22T01:15:17.008 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.016 INFO:teuthology.orchestra.run.smithi096.stdout:3871 writesame 0xba469dd thru 0xba4a4a9 data_size 0x1(0x3acd bytes) 2024-09-22T01:15:17.016 INFO:teuthology.orchestra.run.smithi096.stdout:3872 read 0x5a7d663 thru 0x5a8bb0b (0xe4a9 bytes) 2024-09-22T01:15:17.016 INFO:teuthology.orchestra.run.smithi096.stdout:3873 punch from 0x4e8220a to 0x4e8bc6f, (0x9a65 bytes) 2024-09-22T01:15:17.016 INFO:teuthology.orchestra.run.smithi096.stdout:3874 writesame 0x9f1adf5 thru 0x9f26602 data_size 0x3(0xb80e bytes) 2024-09-22T01:15:17.029 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.029 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.029 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:17.030 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:17.030 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:17.026+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:17.039 INFO:teuthology.orchestra.run.smithi096.stdout:3878 trunc from 0xba4a4aa to 0x480dfc4 2024-09-22T01:15:17.049 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:17.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:17.055 INFO:teuthology.orchestra.run.smithi096.stdout:3879 write 0x22f6cc5 thru 0x2300d77 (0xa0b3 bytes) 2024-09-22T01:15:17.060 INFO:teuthology.orchestra.run.smithi096.stdout:3880 read 0xbcedfb thru 0xbdaa7f (0xbc85 bytes) 2024-09-22T01:15:17.076 INFO:teuthology.orchestra.run.smithi096.stdout:3882 writesame 0xa340895 thru 0xa3412a2 data_size 0xb(0xa0e bytes) 2024-09-22T01:15:17.076 INFO:teuthology.orchestra.run.smithi096.stdout:3883 punch from 0x99acfaf to 0x99ad2a0, (0x2f1 bytes) 2024-09-22T01:15:17.076 INFO:teuthology.orchestra.run.smithi096.stdout:3884 read 0x9354a6b thru 0x936129e (0xc834 bytes) 2024-09-22T01:15:17.081 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:17.090 INFO:teuthology.orchestra.run.smithi096.stdout:3885 writesame 0xe1ca3a8 thru 0xe1cd7d4 data_size 0x13(0x342d bytes) 2024-09-22T01:15:17.090 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.090 INFO:teuthology.orchestra.run.smithi096.stdout:3887 write 0x9f2cced thru 0x9f36359 (0x966d bytes) 2024-09-22T01:15:17.109 INFO:teuthology.orchestra.run.smithi096.stdout:3890 trunc from 0xe1cd7d5 to 0x445595a 2024-09-22T01:15:17.111 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:17.111 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:22366 3.c 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 3932160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.844783071999998, 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022689654, 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 22366 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285078+0000", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.285080+0000", 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.781e-06 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289468+0000", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043884040000000003 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289523+0000", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5278999999999997e-05 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289551+0000", 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7574999999999999e-05 2024-09-22T01:15:17.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289705+0000", 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015408899999999999 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.289922+0000", 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021745600000000001 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:17.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.290362+0000", 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043983999999999999 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.305662+0000", 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.015299975 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307559+0000", 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018972259999999999 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307658+0000", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8330000000000002e-05 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:31.307767+0000", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010969899999999999 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105115 3.3 3:c50281dc:::rbd_data.1124772a770f.000000000000000f:1f [read 481280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.018938+0000", 2024-09-22T01:15:17.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1109233009999997, 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021860435000000001, 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105115 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018938+0000", 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018938+0000", 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018938+0000", 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9200000000000002e-07 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018940+0000", 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018948+0000", 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8259999999999993e-06 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018953+0000", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5620000000000003e-06 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040666+0000", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021712487999999999 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040738+0000", 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1789e-05 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040798+0000", 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0331000000000002e-05 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105117 3.3 3:c50281dc:::rbd_data.1124772a770f.000000000000000f:1f [read 497664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.018960+0000", 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1109011759999996, 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021992345999999999, 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105117 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018960+0000", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018960+0000", 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018960+0000", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4300000000000003e-07 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018962+0000", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9180000000000001e-06 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018970+0000", 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0059999999999992e-06 2024-09-22T01:15:17.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018976+0000", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5829999999999997e-06 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040824+0000", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021848617000000001 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040895+0000", 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0796000000000005e-05 2024-09-22T01:15:17.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040952+0000", 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6882999999999997e-05 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105119 3.3 3:c50281dc:::rbd_data.1124772a770f.000000000000000f:1f [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.018982+0000", 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1108787250000001, 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022117982000000001, 2024-09-22T01:15:17.146 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105119 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018982+0000", 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018982+0000", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018982+0000", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018985+0000", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1670000000000002e-06 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018993+0000", 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8709999999999997e-06 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.018998+0000", 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2870000000000004e-06 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.040978+0000", 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021980025 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041045+0000", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7174000000000004e-05 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041100+0000", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5068000000000002e-05 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105167 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 266240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019343+0000", 2024-09-22T01:15:17.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.110518109, 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021917623000000001, 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105167 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019343+0000", 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019343+0000", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019343+0000", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3799999999999997e-07 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019345+0000", 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9700000000000002e-06 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019351+0000", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2779999999999999e-06 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019356+0000", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.758e-06 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041119+0000", 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021762275000000001 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041194+0000", 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5452999999999997e-05 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041260+0000", 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6351000000000002e-05 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105168 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019362+0000", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1104983160000002, 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022075713, 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105168 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019362+0000", 2024-09-22T01:15:17.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019362+0000", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019363+0000", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019365+0000", 2024-09-22T01:15:17.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7689999999999999e-06 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019371+0000", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000004e-06 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019375+0000", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.357e-06 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041295+0000", 2024-09-22T01:15:17.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021920279000000001 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041372+0000", 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6156999999999996e-05 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041438+0000", 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6589999999999998e-05 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105169 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 299008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019382+0000", 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1104790800000002, 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022233276999999999, 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105169 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019382+0000", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019382+0000", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019382+0000", 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5199999999999998e-07 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019384+0000", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0269999999999998e-06 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019390+0000", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3679999999999998e-06 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019395+0000", 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.831e-06 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041480+0000", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022084275 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041551+0000", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1829999999999995e-05 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041615+0000", 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3594000000000001e-05 2024-09-22T01:15:17.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105170 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019401+0000", 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1104596689999999, 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022423385000000001, 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105170 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019401+0000", 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019401+0000", 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019401+0000", 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019403+0000", 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.832e-06 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019410+0000", 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7209999999999997e-06 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019415+0000", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8450000000000002e-06 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041652+0000", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022237442 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041722+0000", 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9453000000000001e-05 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041825+0000", 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102756 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105171 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019422+0000", 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1104389189999999, 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022576253000000001, 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105171 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019422+0000", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019422+0000", 2024-09-22T01:15:17.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019422+0000", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019425+0000", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2680000000000001e-06 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019431+0000", 2024-09-22T01:15:17.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6250000000000001e-06 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019436+0000", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9930000000000003e-06 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041863+0000", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022427309999999999 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041936+0000", 2024-09-22T01:15:17.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2050000000000003e-05 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.041998+0000", 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2634999999999999e-05 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105172 3.15 3:a880a386:::rbd_data.1124772a770f.000000000000000e:1f [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019442+0000", 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1104185439999998, 2024-09-22T01:15:17.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022748719000000001, 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105172 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019442+0000", 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019442+0000", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019443+0000", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.77e-07 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019445+0000", 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1830000000000001e-06 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019451+0000", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2890000000000003e-06 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019456+0000", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9309999999999999e-06 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042025+0000", 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022568777000000002 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042096+0000", 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1260999999999994e-05 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042191+0000", 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4901000000000005e-05 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105173 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019462+0000", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1103984220000003, 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022780999, 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105173 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019462+0000", 2024-09-22T01:15:17.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019462+0000", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019463+0000", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.84e-07 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019465+0000", 2024-09-22T01:15:17.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6139999999999999e-06 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019472+0000", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5930000000000002e-06 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019477+0000", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042231+0000", 2024-09-22T01:15:17.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022753736 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042243+0000", 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2187e-05 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105174 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019484+0000", 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1103768040000004, 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022796889000000001, 2024-09-22T01:15:17.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105174 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019484+0000", 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019484+0000", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019484+0000", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019486+0000", 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.874e-06 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019493+0000", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5270000000000004e-06 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019497+0000", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2019999999999997e-06 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042269+0000", 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022771851999999999 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042281+0000", 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2072000000000001e-05 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105175 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019503+0000", 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1103578719999998, 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022807738000000001, 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105175 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019503+0000", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019503+0000", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019503+0000", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019506+0000", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019511+0000", 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019515+0000", 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999997e-06 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042300+0000", 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022784887 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042311+0000", 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0338e-05 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105176 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019522+0000", 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1103389530000003, 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022839446999999999, 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105176 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019522+0000", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019522+0000", 2024-09-22T01:15:17.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019522+0000", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019527+0000", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9139999999999999e-06 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019535+0000", 2024-09-22T01:15:17.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7039999999999994e-06 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019540+0000", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4249999999999997e-06 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042349+0000", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022808610999999999 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042361+0000", 2024-09-22T01:15:17.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2381999999999999e-05 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105177 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019548+0000", 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1103132860000002, 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.02285191, 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105177 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019548+0000", 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019548+0000", 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019548+0000", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3000000000000001e-07 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019550+0000", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1399999999999998e-06 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019557+0000", 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8929999999999999e-06 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019562+0000", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3480000000000003e-06 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042389+0000", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022826869 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042399+0000", 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-05 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105178 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019569+0000", 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1102918190000004, 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022875372000000001, 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105178 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.195 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019569+0000", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019569+0000", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019570+0000", 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-22T01:15:17.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019571+0000", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8899999999999999e-06 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019578+0000", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6680000000000004e-06 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019586+0000", 2024-09-22T01:15:17.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3769999999999996e-06 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042434+0000", 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022847263 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042444+0000", 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0606e-05 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105179 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019597+0000", 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1102634130000002, 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022887613000000001, 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105179 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019597+0000", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019597+0000", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019598+0000", 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-22T01:15:17.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019600+0000", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019607+0000", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019612+0000", 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0270000000000003e-06 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042476+0000", 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022864569000000001 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042485+0000", 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8429999999999999e-06 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105180 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019618+0000", 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1102424170000003, 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022912464, 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105180 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019618+0000", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019618+0000", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019619+0000", 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3900000000000004e-07 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019621+0000", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3120000000000001e-06 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019628+0000", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.455e-06 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019633+0000", 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9779999999999996e-06 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042519+0000", 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022886472000000001 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042531+0000", 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1608e-05 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105181 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019639+0000", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1102215559999999, 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022910040999999999, 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105181 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019639+0000", 2024-09-22T01:15:17.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019639+0000", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019640+0000", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-07 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019642+0000", 2024-09-22T01:15:17.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1280000000000002e-06 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019648+0000", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2820000000000002e-06 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019653+0000", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042545+0000", 2024-09-22T01:15:17.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022892181000000001 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042549+0000", 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1919999999999998e-06 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:105182 3.15 3.61c50115 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:13.019659+0000", 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1102013599999996, 2024-09-22T01:15:17.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022925619000000001, 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 105182 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019659+0000", 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019659+0000", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019660+0000", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5400000000000002e-07 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019662+0000", 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.119e-06 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.019669+0000", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.024593+0000", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049242970000000002 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042576+0000", 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.017982921999999998 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:13.042585+0000", 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2939999999999994e-06 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.214 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:17.214 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:17.125+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:17.215 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:17.215 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:17.215 INFO:teuthology.orchestra.run.smithi096.stdout:3892 write 0x82061a6 thru 0x820b476 (0x52d1 bytes) 2024-09-22T01:15:17.215 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.215 INFO:teuthology.orchestra.run.smithi096.stdout:3894 punch from 0x14660bd to 0x1471587, (0xb4ca bytes) 2024-09-22T01:15:17.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:17.217 INFO:teuthology.orchestra.run.smithi096.stdout:3895 write 0xa7181e8 thru 0xa71fb7c (0x7995 bytes) 2024-09-22T01:15:17.217 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:17.217 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:17.218 INFO:teuthology.orchestra.run.smithi096.stdout:3897 read 0x19dbf8f thru 0x19e90b4 (0xd126 bytes) 2024-09-22T01:15:17.218 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3900 read 0xa18e82 thru 0xa193c8 (0x547 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3901 punch from 0x1d597ad to 0x1d645e9, (0xae3c bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3902 punch from 0x57c770b to 0x57cc571, (0x4e66 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3903 punch from 0x9884ab6 to 0x9887e5c, (0x33a6 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3904 punch from 0x7e55eef to 0x7e628e8, (0xc9f9 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3906 read 0x3ddce45 thru 0x3de5f36 (0x90f2 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3908 writesame 0x875d11e thru 0x8762061 data_size 0x2(0x4f44 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3910 writesame 0x471e427 thru 0x47274d3 data_size 0x5b(0x90ad bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3911 read 0x93e9b80 thru 0x93eefa5 (0x5426 bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3914 read 0x16afd68 thru 0x16b10e6 (0x137f bytes) 2024-09-22T01:15:17.219 INFO:teuthology.orchestra.run.smithi096.stdout:3916 write 0x2bdf292 thru 0x2be6b4e (0x78bd bytes) 2024-09-22T01:15:17.220 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:17.220 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:17.220 INFO:teuthology.orchestra.run.smithi096.stdout:3920 read 0x16dae7f thru 0x16e8caa (0xde2c bytes) 2024-09-22T01:15:17.220 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.220 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.221 INFO:teuthology.orchestra.run.smithi096.stdout:3923 trunc from 0xa71fb7d to 0x889839 2024-09-22T01:15:17.224 INFO:teuthology.orchestra.run.smithi096.stdout:3924 read 0x4a9372 thru 0x4b8a72 (0xf701 bytes) 2024-09-22T01:15:17.232 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:17.226+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:3925 write 0x473265f thru 0x473fbc6 (0xd568 bytes) 2024-09-22T01:15:17.241 INFO:teuthology.orchestra.run.smithi096.stdout:3927 punch from 0x3457246 to 0x345a958, (0x3712 bytes) 2024-09-22T01:15:17.251 INFO:teuthology.orchestra.run.smithi096.stdout:3928 write 0x6a21af3 thru 0x6a2239f (0x8ad bytes) 2024-09-22T01:15:17.254 INFO:teuthology.orchestra.run.smithi096.stdout:3929 punch from 0xd03482 to 0xd0f78e, (0xc30c bytes) 2024-09-22T01:15:17.270 INFO:teuthology.orchestra.run.smithi096.stdout:3930 write 0xceaaf7e thru 0xceba176 (0xf1f9 bytes) 2024-09-22T01:15:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:3931 writesame 0xe4bfa04 thru 0xe4cb6c6 data_size 0x17(0xbcc3 bytes) 2024-09-22T01:15:17.285 INFO:teuthology.orchestra.run.smithi096.stdout:3933 read 0xd9ccc31 thru 0xd9db60d (0xe9dd bytes) 2024-09-22T01:15:17.292 INFO:teuthology.orchestra.run.smithi096.stdout:3934 trunc from 0xe4cb6c7 to 0x98caebe 2024-09-22T01:15:17.306 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.317 INFO:teuthology.orchestra.run.smithi096.stdout:3936 writesame 0xc17d1e3 thru 0xc183de1 data_size 0x1(0x6bff bytes) 2024-09-22T01:15:17.317 INFO:teuthology.orchestra.run.smithi096.stdout:3937 punch from 0x7d868d7 to 0x7d86c88, (0x3b1 bytes) 2024-09-22T01:15:17.327 INFO:teuthology.orchestra.run.smithi096.stdout:3939 trunc from 0xc183de2 to 0x497b1b0 2024-09-22T01:15:17.333 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:17.326+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:17.349 INFO:teuthology.orchestra.run.smithi096.stdout:3940 write 0x6a59ffd thru 0x6a5cf8a (0x2f8e bytes) 2024-09-22T01:15:17.356 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:17.356 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:17.356 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:17.361 INFO:teuthology.orchestra.run.smithi096.stdout:3941 trunc from 0x6a5cf8b to 0x2b467fe 2024-09-22T01:15:17.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:3944 punch from 0x8519f to 0x8d67c, (0x84dd bytes) 2024-09-22T01:15:17.381 INFO:teuthology.orchestra.run.smithi096.stdout:3945 read 0x2252c3e thru 0x2260251 (0xd614 bytes) 2024-09-22T01:15:17.382 INFO:teuthology.orchestra.run.smithi096.stdout:3946 read 0x5ce232 thru 0x5db5fc (0xd3cb bytes) 2024-09-22T01:15:17.383 INFO:teuthology.orchestra.run.smithi096.stdout:3948 writesame 0x1594411 thru 0x15965ab data_size 0x1(0x219b bytes) 2024-09-22T01:15:17.393 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.393 INFO:teuthology.orchestra.run.smithi096.stdout:3951 read 0x2965971 thru 0x296ce0c (0x749c bytes) 2024-09-22T01:15:17.417 INFO:teuthology.orchestra.run.smithi096.stdout:3952 writesame 0xd6abcc0 thru 0xd6afc7a data_size 0xd(0x3fbb bytes) 2024-09-22T01:15:17.417 INFO:teuthology.orchestra.run.smithi096.stdout:3953 read 0x5100ee2 thru 0x51052cc (0x43eb bytes) 2024-09-22T01:15:17.417 INFO:teuthology.orchestra.run.smithi096.stdout:3956 write 0xaec2c39 thru 0xaec7ddc (0x51a4 bytes) 2024-09-22T01:15:17.433 INFO:teuthology.orchestra.run.smithi096.stdout:3957 write 0xe40a62e thru 0xe4129a3 (0x8376 bytes) 2024-09-22T01:15:17.433 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:17.429+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:17.439 INFO:teuthology.orchestra.run.smithi096.stdout:3958 read 0x2581c62 thru 0x258c5d2 (0xa971 bytes) 2024-09-22T01:15:17.439 INFO:teuthology.orchestra.run.smithi096.stdout:3959 write 0x9dea669 thru 0x9df5065 (0xa9fd bytes) 2024-09-22T01:15:17.455 INFO:teuthology.orchestra.run.smithi096.stdout:3960 trunc from 0xe4129a4 to 0x770233b 2024-09-22T01:15:17.473 INFO:teuthology.orchestra.run.smithi096.stdout:3961 read 0x13a0b7d thru 0x13a7bc6 (0x704a bytes) 2024-09-22T01:15:17.477 INFO:teuthology.orchestra.run.smithi096.stdout:3963 trunc from 0x770233b to 0x4db2799 2024-09-22T01:15:17.486 INFO:teuthology.orchestra.run.smithi096.stdout:3965 write 0x4d86313 thru 0x4d8aa37 (0x4725 bytes) 2024-09-22T01:15:17.493 INFO:teuthology.orchestra.run.smithi096.stdout:3966 write 0x1f3129 thru 0x1f37df (0x6b7 bytes) 2024-09-22T01:15:17.513 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:17.513 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:17.513 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:17.513 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:17.513 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:17.519 INFO:teuthology.orchestra.run.smithi096.stdout:3967 write 0x10939d4 thru 0x109890e (0x4f3b bytes) 2024-09-22T01:15:17.528 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:17.541 INFO:teuthology.orchestra.run.smithi096.stdout:3968 write 0xa84d441 thru 0xa851072 (0x3c32 bytes) 2024-09-22T01:15:17.542 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:17.537+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:17.547 INFO:teuthology.orchestra.run.smithi096.stdout:3969 read 0x91feb7f thru 0x9205040 (0x64c2 bytes) 2024-09-22T01:15:17.548 INFO:teuthology.orchestra.run.smithi096.stdout:3970 writesame 0x521c3ca thru 0x521e82f data_size 0x3(0x2466 bytes) 2024-09-22T01:15:17.556 INFO:teuthology.orchestra.run.smithi096.stdout:3972 punch from 0x3f64414 to 0x3f668f3, (0x24df bytes) 2024-09-22T01:15:17.571 INFO:teuthology.orchestra.run.smithi096.stdout:3973 write 0xe12c60c thru 0xe1309b7 (0x43ac bytes) 2024-09-22T01:15:17.577 INFO:teuthology.orchestra.run.smithi096.stdout:3974 write 0xb92a6a1 thru 0xb938464 (0xddc4 bytes) 2024-09-22T01:15:17.583 INFO:teuthology.orchestra.run.smithi096.stdout:3976 punch from 0x789df1b to 0x789e1b8, (0x29d bytes) 2024-09-22T01:15:17.583 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.583 INFO:teuthology.orchestra.run.smithi096.stdout:3978 read 0x5b0539 thru 0x5ba8de (0xa3a6 bytes) 2024-09-22T01:15:17.584 INFO:teuthology.orchestra.run.smithi096.stdout:3979 write 0xd05b460 thru 0xd06a578 (0xf119 bytes) 2024-09-22T01:15:17.592 INFO:teuthology.orchestra.run.smithi096.stdout:3980 read 0xc55f671 thru 0xc564217 (0x4ba7 bytes) 2024-09-22T01:15:17.592 INFO:teuthology.orchestra.run.smithi096.stdout:3982 punch from 0xd4d457e to 0xd4d86ef, (0x4171 bytes) 2024-09-22T01:15:17.592 INFO:teuthology.orchestra.run.smithi096.stdout:3984 write 0x80365a1 thru 0x803d348 (0x6da8 bytes) 2024-09-22T01:15:17.598 INFO:teuthology.orchestra.run.smithi096.stdout:3985 punch from 0xbdcd1b2 to 0xbdd1539, (0x4387 bytes) 2024-09-22T01:15:17.598 INFO:teuthology.orchestra.run.smithi096.stdout:3986 read 0x3be1eae thru 0x3be29d7 (0xb2a bytes) 2024-09-22T01:15:17.598 INFO:teuthology.orchestra.run.smithi096.stdout:3987 punch from 0x687b51a to 0x68849f2, (0x94d8 bytes) 2024-09-22T01:15:17.598 INFO:teuthology.orchestra.run.smithi096.stdout:3988 write 0x79433a5 thru 0x7945664 (0x22c0 bytes) 2024-09-22T01:15:17.602 INFO:teuthology.orchestra.run.smithi096.stdout:3989 read 0xc743315 thru 0xc74b35b (0x8047 bytes) 2024-09-22T01:15:17.602 INFO:teuthology.orchestra.run.smithi096.stdout:3990 writesame 0xce5e789 thru 0xce60f49 data_size 0x1(0x27c1 bytes) 2024-09-22T01:15:17.615 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.615 INFO:teuthology.orchestra.run.smithi096.stdout:3993 read 0x6de7d65 thru 0x6defdcf (0x806b bytes) 2024-09-22T01:15:17.615 INFO:teuthology.orchestra.run.smithi096.stdout:3995 read 0x7d41009 thru 0x7d41d5e (0xd56 bytes) 2024-09-22T01:15:17.615 INFO:teuthology.orchestra.run.smithi096.stdout:3996 write 0x66c27bb thru 0x66c5202 (0x2a48 bytes) 2024-09-22T01:15:17.621 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.621 INFO:teuthology.orchestra.run.smithi096.stdout:4002 punch from 0x1d84ae2 to 0x1d892cf, (0x47ed bytes) 2024-09-22T01:15:17.621 INFO:teuthology.orchestra.run.smithi096.stdout:4004 writesame 0x6e61c92 thru 0x6e70bd9 data_size 0x7c(0xef48 bytes) 2024-09-22T01:15:17.627 INFO:teuthology.orchestra.run.smithi096.stdout:4006 writesame 0x4fb91bd thru 0x4fc595f data_size 0x7(0xc7a3 bytes) 2024-09-22T01:15:17.635 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:17.629+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:17.636 INFO:teuthology.orchestra.run.smithi096.stdout:4007 write 0x64e3cf7 thru 0x64ed983 (0x9c8d bytes) 2024-09-22T01:15:17.642 INFO:teuthology.orchestra.run.smithi096.stdout:4008 read 0x1416f62 thru 0x142339f (0xc43e bytes) 2024-09-22T01:15:17.643 INFO:teuthology.orchestra.run.smithi096.stdout:4009 punch from 0x569bb84 to 0x56a2cf4, (0x7170 bytes) 2024-09-22T01:15:17.644 INFO:teuthology.orchestra.run.smithi096.stdout:4011 trunc from 0xe1309b8 to 0xd0902dd 2024-09-22T01:15:17.670 INFO:teuthology.orchestra.run.smithi096.stdout:4012 read 0xacf0ad4 thru 0xacfce0a (0xc337 bytes) 2024-09-22T01:15:17.671 INFO:teuthology.orchestra.run.smithi096.stdout:4013 writesame 0x9e30705 thru 0x9e3f84a data_size 0x2(0xf146 bytes) 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:17.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.687498234000003, 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:17.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:15:17.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57902 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4516864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481772805999999, 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055853368, 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57902 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190194+0000", 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:15:17.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190196+0000", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190202+0000", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1519999999999997e-06 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190207+0000", 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8960000000000002e-06 2024-09-22T01:15:17.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245857+0000", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055649864 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245932+0000", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4938999999999995e-05 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246047+0000", 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115036 2024-09-22T01:15:17.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481750575, 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:15:17.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:17.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481717802999999, 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:17.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:15:17.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481696338999999, 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481675396, 2024-09-22T01:15:17.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:15:17.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:15:17.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481655488000001, 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481635106999999, 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481613169999999, 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:15:17.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:15:17.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481103589, 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.749722542000001, 2024-09-22T01:15:17.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:17.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.749688238000001, 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:17.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:15:17.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:15:17.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.749617594, 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.74946589, 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:15:17.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:15:17.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:15:17.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.749250353000001, 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.749037128000001, 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:15:17.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:15:17.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.748839004000001, 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:15:17.742 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:17.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.748635212, 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:15:17.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:15:17.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.748384387000002, 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.748165366999999, 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:17.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:17.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:17.759 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:17.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:17.761 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.761 INFO:teuthology.orchestra.run.smithi096.stdout:4015 punch from 0xbf6eda4 to 0xbf70e37, (0x2093 bytes) 2024-09-22T01:15:17.761 INFO:teuthology.orchestra.run.smithi096.stdout:4016 write 0x9b7e52a thru 0x9b86b4d (0x8624 bytes) 2024-09-22T01:15:17.761 INFO:teuthology.orchestra.run.smithi096.stdout:4017 writesame 0x56d5606 thru 0x56de465 data_size 0x86(0x8e60 bytes) 2024-09-22T01:15:17.761 INFO:teuthology.orchestra.run.smithi096.stdout:4018 trunc from 0xd0902dd to 0x2bd69f0 2024-09-22T01:15:17.761 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:17.754+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:17.768 INFO:teuthology.orchestra.run.smithi096.stdout:4020 read 0x229b78d thru 0x229df45 (0x27b9 bytes) 2024-09-22T01:15:17.768 INFO:teuthology.orchestra.run.smithi096.stdout:4023 write 0x90fa7d thru 0x911d36 (0x22ba bytes) 2024-09-22T01:15:17.770 INFO:teuthology.orchestra.run.smithi096.stdout:4025 read 0x19b00f4 thru 0x19bb444 (0xb351 bytes) 2024-09-22T01:15:17.790 INFO:teuthology.orchestra.run.smithi096.stdout:4026 write 0x8979050 thru 0x897bdf5 (0x2da6 bytes) 2024-09-22T01:15:17.816 INFO:teuthology.orchestra.run.smithi096.stdout:4027 write 0xea3b607 thru 0xea3e514 (0x2f0e bytes) 2024-09-22T01:15:17.820 INFO:teuthology.orchestra.run.smithi096.stdout:4028 write 0x2340ec1 thru 0x234d049 (0xc189 bytes) 2024-09-22T01:15:17.826 INFO:teuthology.orchestra.run.smithi096.stdout:4029 writesame 0x74bbaff thru 0x74c1bea data_size 0x2(0x60ec bytes) 2024-09-22T01:15:17.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:15:17.843 INFO:teuthology.orchestra.run.smithi096.stdout:4030 writesame 0x28d2c7c thru 0x28e0c9b data_size 0x16(0xe020 bytes) 2024-09-22T01:15:17.851 INFO:teuthology.orchestra.run.smithi096.stdout:4032 punch from 0x22a70c1 to 0x22b2611, (0xb550 bytes) 2024-09-22T01:15:17.851 INFO:teuthology.orchestra.run.smithi096.stdout:4034 punch from 0x167cd64 to 0x168b6a4, (0xe940 bytes) 2024-09-22T01:15:17.861 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:17.857+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:17.871 INFO:teuthology.orchestra.run.smithi096.stdout:4036 trunc from 0xea3e515 to 0x11348c2 2024-09-22T01:15:17.907 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:17.907 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:17.907 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:17.907 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:17.919 INFO:teuthology.orchestra.run.smithi096.stdout:4038 write 0xbcffd4d thru 0xbd004ef (0x7a3 bytes) 2024-09-22T01:15:17.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:17.928 INFO:teuthology.orchestra.run.smithi096.stdout:4039 punch from 0x7c4fead to 0x7c57d6b, (0x7ebe bytes) 2024-09-22T01:15:17.928 INFO:teuthology.orchestra.run.smithi096.stdout:4042 writesame 0xa0ea695 thru 0xa0ee166 data_size 0x1(0x3ad2 bytes) 2024-09-22T01:15:17.940 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:17.940 INFO:teuthology.orchestra.run.smithi096.stdout:4045 read 0xa1ecebe thru 0xa1f7f0a (0xb04d bytes) 2024-09-22T01:15:17.941 INFO:teuthology.orchestra.run.smithi096.stdout:4046 punch from 0x7ab6802 to 0x7ab7dea, (0x15e8 bytes) 2024-09-22T01:15:17.942 INFO:teuthology.orchestra.run.smithi096.stdout:4047 read 0xbcc0ae6 thru 0xbcc4ddf (0x42fa bytes) 2024-09-22T01:15:17.942 INFO:teuthology.orchestra.run.smithi096.stdout:4048 punch from 0x2ab002c to 0x2ab04a4, (0x478 bytes) 2024-09-22T01:15:17.947 INFO:teuthology.orchestra.run.smithi096.stdout:4049 trunc from 0xbd004f0 to 0x7e41603 2024-09-22T01:15:17.962 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:17.958+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:17.963 INFO:teuthology.orchestra.run.smithi096.stdout:4051 write 0x6aa9cc0 thru 0x6ab495f (0xaca0 bytes) 2024-09-22T01:15:17.968 INFO:teuthology.orchestra.run.smithi096.stdout:4052 punch from 0x6f924c9 to 0x6f9e6b0, (0xc1e7 bytes) 2024-09-22T01:15:17.968 INFO:teuthology.orchestra.run.smithi096.stdout:4054 write 0x465df4b thru 0x465e452 (0x508 bytes) 2024-09-22T01:15:17.972 INFO:teuthology.orchestra.run.smithi096.stdout:4055 writesame 0xa2662c thru 0xa2aa91 data_size 0x55(0x4466 bytes) 2024-09-22T01:15:17.976 INFO:teuthology.orchestra.run.smithi096.stdout:4056 read 0x6d59e9 thru 0x6dd429 (0x7a41 bytes) 2024-09-22T01:15:17.977 INFO:teuthology.orchestra.run.smithi096.stdout:4057 read 0x1220713 thru 0x1223d0f (0x35fd bytes) 2024-09-22T01:15:17.977 INFO:teuthology.orchestra.run.smithi096.stdout:4058 clone 34 order 24 su 1048576 sc 2 2024-09-22T01:15:18.050 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:18.050 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:18.050 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:18.051 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:18.051 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:18.063 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:18.062+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:18.064 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:18.163 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:18.158+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:18.226 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:18.226 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:18.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:18.226 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:18.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.327722992999998, 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.327558636999999, 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:15:18.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322866861000001, 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:18.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322829306999999, 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:18.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:15:18.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:15:18.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322797144999999, 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322774807999998, 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:15:18.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:15:18.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322753448, 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:15:18.252 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.322732257999998, 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:18.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:15:18.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318826765000001, 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:15:18.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:15:18.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:15:18.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318804583999999, 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318783306, 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:18.273 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318760958999999, 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:15:18.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:15:18.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318739282999999, 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.318717861, 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.312079116, 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.311862369, 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:15:18.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:15:18.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.311479742, 2024-09-22T01:15:18.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:15:18.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:15:18.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.311074202, 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.307857740999999, 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:15:18.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:18.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:15:18.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.271072346, 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:15:18.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.311 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:18.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:18.313 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:18.310+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:18.414 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:18.409+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:18.437 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:18.437 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:18.437 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:18.437 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:18.451 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:18.515 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:18.514+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:18.613 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:18.614 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:18.614 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:18.614 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:18.614 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:18.615 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:18.609+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:18.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:18.716 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:18.713+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.775891252999998, 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 44.228327456999999, 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.217016792000003, 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.978665707999999, 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487902339999998, 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487792129999992, 2024-09-22T01:15:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487683890000007, 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487575099999994, 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487434969999999, 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:18.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:18.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:18.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6487125720000009, 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:18.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:18.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6486874839999999, 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:18.846 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.850 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6486632490000002, 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6486412329999993, 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:18.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.859 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6486117240000002, 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:18.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6485600389999995, 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:18.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6485274099999998, 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:18.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6484700320000005, 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:18.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:18.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6484549679999994, 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:18.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:18.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6484420439999994, 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.6484304450000007, 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:18.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:18.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:18.891 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:18.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:18.894 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:15:18.894 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:18.889+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:18.937 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone33 from 0x7e41603 (overlap 0x889839) to 0x28fa77 2024-09-22T01:15:18.994 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:18.990+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:19.031 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:19.031 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:19.031 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:19.031 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:19.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:19.095 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:19.093+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:19.096 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-22T01:15:19.120 INFO:teuthology.orchestra.run.smithi096.stdout:4059 write 0x4441467 thru 0x444876c (0x7306 bytes) 2024-09-22T01:15:19.137 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.137 INFO:teuthology.orchestra.run.smithi096.stdout:4061 punch from 0x399b4f5 to 0x39a7b83, (0xc68e bytes) 2024-09-22T01:15:19.137 INFO:teuthology.orchestra.run.smithi096.stdout:4062 read 0x50925be thru 0x5098c98 (0x66db bytes) 2024-09-22T01:15:19.138 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.138 INFO:teuthology.orchestra.run.smithi096.stdout:4065 read 0x1b51302 thru 0x1b54f67 (0x3c66 bytes) 2024-09-22T01:15:19.138 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.150 INFO:teuthology.orchestra.run.smithi096.stdout:4069 write 0x8dec620 thru 0x8deff1c (0x38fd bytes) 2024-09-22T01:15:19.157 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.158 INFO:teuthology.orchestra.run.smithi096.stdout:4071 read 0x4390112 thru 0x439e28b (0xe17a bytes) 2024-09-22T01:15:19.173 INFO:teuthology.orchestra.run.smithi096.stdout:4072 write 0x984cf3c thru 0x9852adb (0x5ba0 bytes) 2024-09-22T01:15:19.180 INFO:teuthology.orchestra.run.smithi096.stdout:4074 write 0x52a1d6a thru 0x52af911 (0xdba8 bytes) 2024-09-22T01:15:19.198 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:19.194+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:19.201 INFO:teuthology.orchestra.run.smithi096.stdout:4075 writesame 0xa147a8e thru 0xa147ed3 data_size 0x1(0x446 bytes) 2024-09-22T01:15:19.212 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:19.212 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:19.213 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:19.213 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:19.213 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:19.222 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:19.222 INFO:teuthology.orchestra.run.smithi096.stdout:4076 trunc from 0xa147ed4 to 0x26b956f 2024-09-22T01:15:19.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:19.265 INFO:teuthology.orchestra.run.smithi096.stdout:4077 trunc from 0x26b956f to 0x7779e37 2024-09-22T01:15:19.273 INFO:teuthology.orchestra.run.smithi096.stdout:4078 write 0x45b6e25 thru 0x45c1df8 (0xafd4 bytes) 2024-09-22T01:15:19.280 INFO:teuthology.orchestra.run.smithi096.stdout:4079 write 0x17e79da thru 0x17eb456 (0x3a7d bytes) 2024-09-22T01:15:19.299 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:19.293+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:19.310 INFO:teuthology.orchestra.run.smithi096.stdout:4080 write 0x97c6b0 thru 0x98c09a (0xf9eb bytes) 2024-09-22T01:15:19.312 INFO:teuthology.orchestra.run.smithi096.stdout:4081 read 0x2742de3 thru 0x2752b22 (0xfd40 bytes) 2024-09-22T01:15:19.313 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.313 INFO:teuthology.orchestra.run.smithi096.stdout:4084 read 0x4de5573 thru 0x4de98b2 (0x4340 bytes) 2024-09-22T01:15:19.314 INFO:teuthology.orchestra.run.smithi096.stdout:4085 punch from 0x5b65a32 to 0x5b67cdf, (0x22ad bytes) 2024-09-22T01:15:19.314 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.314 INFO:teuthology.orchestra.run.smithi096.stdout:4088 punch from 0x6cf0d0a to 0x6cfb5ce, (0xa8c4 bytes) 2024-09-22T01:15:19.314 INFO:teuthology.orchestra.run.smithi096.stdout:4089 punch from 0x4b5bc05 to 0x4b624f9, (0x68f4 bytes) 2024-09-22T01:15:19.322 INFO:teuthology.orchestra.run.smithi096.stdout:4090 trunc from 0x7779e37 to 0xc6b13c6 2024-09-22T01:15:19.344 INFO:teuthology.orchestra.run.smithi096.stdout:4093 trunc from 0xc6b13c6 to 0x446eb51 2024-09-22T01:15:19.355 INFO:teuthology.orchestra.run.smithi096.stdout:4094 write 0x3f02b27 thru 0x3f0b3e2 (0x88bc bytes) 2024-09-22T01:15:19.377 INFO:teuthology.orchestra.run.smithi096.stdout:4095 write 0xa84ca12 thru 0xa84ee21 (0x2410 bytes) 2024-09-22T01:15:19.382 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:19.382 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.039066191000003, 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:19.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.039035921, 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.039006348000001, 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.403 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:19.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:19.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:19.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038952256999998, 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038896413000003, 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:19.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:19.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038838523000003, 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.03877679, 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.426 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038681326999999, 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.427 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:19.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038619504000003, 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:19.431 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038528364000001, 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.436 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:19.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038466458000002, 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038405971000003, 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038348501000002, 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:19.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:19.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038291221000001, 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038231697000001, 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.038178813000002, 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:19.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.029066251000003, 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:19.466 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:19.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:19.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.585807065000001, 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:19.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:19.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.564735284999999, 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:19.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:19.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:19.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.3784177260000003, 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:19.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:19.485 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:19.486 INFO:teuthology.orchestra.run.smithi096.stdout:4096 trunc from 0xa84ee22 to 0x3a97bbd 2024-09-22T01:15:19.486 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:19.487 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:19.487 INFO:teuthology.orchestra.run.smithi096.stdout:4097 read 0x39d5826 thru 0x39d9393 (0x3b6e bytes) 2024-09-22T01:15:19.487 INFO:teuthology.orchestra.run.smithi096.stdout:4098 write 0x7bfa27 thru 0x7cf90d (0xfee7 bytes) 2024-09-22T01:15:19.488 INFO:teuthology.orchestra.run.smithi096.stdout:4099 punch from 0x2372569 to 0x237528d, (0x2d24 bytes) 2024-09-22T01:15:19.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:19.488 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:19.489 INFO:teuthology.orchestra.run.smithi096.stdout:4100 write 0x8c8a2d9 thru 0x8c8bb41 (0x1869 bytes) 2024-09-22T01:15:19.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:4101 write 0xc276ad4 thru 0xc282ec0 (0xc3ed bytes) 2024-09-22T01:15:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:4103 punch from 0x4ce76a5 to 0x4cea57e, (0x2ed9 bytes) 2024-09-22T01:15:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:4104 write 0x6d62e84 thru 0x6d6d3a7 (0xa524 bytes) 2024-09-22T01:15:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:4106 write 0x7a58600 thru 0x7a5d7ac (0x51ad bytes) 2024-09-22T01:15:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:19.491 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:19.491 INFO:teuthology.orchestra.run.smithi096.stdout:4109 writesame 0xa55d475 thru 0xa5668f9 data_size 0x1(0x9485 bytes) 2024-09-22T01:15:19.491 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:19.481+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:19.491 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:19.492 INFO:teuthology.orchestra.run.smithi096.stdout:4110 read 0x2df3257 thru 0x2dfb2f3 (0x809d bytes) 2024-09-22T01:15:19.507 INFO:teuthology.orchestra.run.smithi096.stdout:4111 trunc from 0xc282ec1 to 0x38e0fec 2024-09-22T01:15:19.512 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:19.512 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:19.524 INFO:teuthology.orchestra.run.smithi096.stdout:4112 trunc from 0x38e0fec to 0x3529f1e 2024-09-22T01:15:19.533 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:19.533 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:19.545 INFO:teuthology.orchestra.run.smithi096.stdout:4113 write 0x6c4aa97 thru 0x6c56a6d (0xbfd7 bytes) 2024-09-22T01:15:19.550 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.550 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.550 INFO:teuthology.orchestra.run.smithi096.stdout:4116 write 0x1f80df6 thru 0x1f8efd8 (0xe1e3 bytes) 2024-09-22T01:15:19.552 INFO:teuthology.orchestra.run.smithi096.stdout:4117 write 0x3add501 thru 0x3adf8c0 (0x23c0 bytes) 2024-09-22T01:15:19.555 INFO:teuthology.orchestra.run.smithi096.stdout:4118 write 0x24edaf3 thru 0x24fb847 (0xdd55 bytes) 2024-09-22T01:15:19.557 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.557 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:19.561 INFO:teuthology.orchestra.run.smithi096.stdout:4121 trunc from 0x6c56a6e to 0x99b1a77 2024-09-22T01:15:19.574 INFO:teuthology.orchestra.run.smithi096.stdout:4122 write 0xa5eef17 thru 0xa5f3ef2 (0x4fdc bytes) 2024-09-22T01:15:19.579 INFO:teuthology.orchestra.run.smithi096.stdout:4123 writesame 0x5d0f494 thru 0x5d1bc6a data_size 0x1(0xc7d7 bytes) 2024-09-22T01:15:19.587 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:19.581+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:19.611 INFO:teuthology.orchestra.run.smithi096.stdout:4124 read 0x3f64aa9 thru 0x3f65577 (0xacf bytes) 2024-09-22T01:15:19.629 INFO:teuthology.orchestra.run.smithi096.stdout:4125 write 0xd6e71a2 thru 0xd6ecda9 (0x5c08 bytes) 2024-09-22T01:15:19.629 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:19.630 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:19.637 INFO:teuthology.orchestra.run.smithi096.stdout:4126 write 0xafefe01 thru 0xaffb1cf (0xb3cf bytes) 2024-09-22T01:15:19.641 INFO:teuthology.orchestra.run.smithi096.stdout:4128 clone 35 order 21 su 262144 sc 12 2024-09-22T01:15:19.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:19.690 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:19.686+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:19.765 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:19.765 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:19.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:19.822 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:19.818+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:19.889 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:19.882+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:19.949 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:19.949 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:19.963 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:19.999 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone34 from 0xd6ecdaa (overlap 0x26b956f) to 0xb9e007 2024-09-22T01:15:20.006 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:20.002+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:20.090 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:20.085+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:20.132 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.132 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:20.191 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:20.185+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:20.243 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-22T01:15:20.256 INFO:teuthology.orchestra.run.smithi096.stdout:4130 read 0x1499313 thru 0x14a7b6f (0xe85d bytes) 2024-09-22T01:15:20.258 INFO:teuthology.orchestra.run.smithi096.stdout:4132 read 0xd7e320 thru 0xd8b1bd (0xce9e bytes) 2024-09-22T01:15:20.274 INFO:teuthology.orchestra.run.smithi096.stdout:4134 writesame 0xdf56c13 thru 0xdf62631 data_size 0x1f(0xba1f bytes) 2024-09-22T01:15:20.274 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.274 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.288 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:20.290 INFO:teuthology.orchestra.run.smithi096.stdout:4135 trunc from 0xdf62632 to 0x5b2572e 2024-09-22T01:15:20.291 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:20.285+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:20.335 INFO:teuthology.orchestra.run.smithi096.stdout:4136 read 0x3ffd90b thru 0x3fffbf8 (0x22ee bytes) 2024-09-22T01:15:20.335 INFO:teuthology.orchestra.run.smithi096.stdout:4137 read 0x48c997b thru 0x48d49e4 (0xb06a bytes) 2024-09-22T01:15:20.336 INFO:teuthology.orchestra.run.smithi096.stdout:4138 read 0x483fbe4 thru 0x4849512 (0x992f bytes) 2024-09-22T01:15:20.337 INFO:teuthology.orchestra.run.smithi096.stdout:4140 write 0x43819db thru 0x4385eb3 (0x44d9 bytes) 2024-09-22T01:15:20.343 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:20.343 INFO:teuthology.orchestra.run.smithi096.stdout:4144 write 0x488b526 thru 0x489766a (0xc145 bytes) 2024-09-22T01:15:20.369 INFO:teuthology.orchestra.run.smithi096.stdout:4145 writesame 0xcdaead0 thru 0xcdb8cd3 data_size 0x2(0xa204 bytes) 2024-09-22T01:15:20.369 INFO:teuthology.orchestra.run.smithi096.stdout:4146 punch from 0xa610348 to 0xa610ed5, (0xb8d bytes) 2024-09-22T01:15:20.369 INFO:teuthology.orchestra.run.smithi096.stdout:4148 punch from 0x1cad98 to 0x1d7dd2, (0xd03a bytes) 2024-09-22T01:15:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:4149 trunc from 0xcdb8cd4 to 0xbbe2775 2024-09-22T01:15:20.381 INFO:teuthology.orchestra.run.smithi096.stdout:4150 write 0xb6b9447 thru 0xb6bd9da (0x4594 bytes) 2024-09-22T01:15:20.387 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:20.387 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:20.387 INFO:teuthology.orchestra.run.smithi096.stdout:4154 read 0x47cb3f0 thru 0x47cde33 (0x2a44 bytes) 2024-09-22T01:15:20.389 INFO:teuthology.orchestra.run.smithi096.stdout:4155 read 0xb70cf23 thru 0xb71797b (0xaa59 bytes) 2024-09-22T01:15:20.389 INFO:teuthology.orchestra.run.smithi096.stdout:4156 punch from 0x6ac66e3 to 0x6ac8bad, (0x24ca bytes) 2024-09-22T01:15:20.389 INFO:teuthology.orchestra.run.smithi096.stdout:4157 writesame 0x5b646a3 thru 0x5b6ceb3 data_size 0x11(0x8811 bytes) 2024-09-22T01:15:20.391 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:20.386+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:20.394 INFO:teuthology.orchestra.run.smithi096.stdout:4158 writesame 0x400e66c thru 0x4017f4b data_size 0x20(0x98e0 bytes) 2024-09-22T01:15:20.403 INFO:teuthology.orchestra.run.smithi096.stdout:4159 read 0xabca78f thru 0xabd5a1c (0xb28e bytes) 2024-09-22T01:15:20.403 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:20.404 INFO:teuthology.orchestra.run.smithi096.stdout:4161 trunc from 0xbbe2775 to 0xc137e96 2024-09-22T01:15:20.410 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.410 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.425 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:20.439 INFO:teuthology.orchestra.run.smithi096.stdout:4162 trunc from 0xc137e96 to 0x2a982ff 2024-09-22T01:15:20.492 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:20.489+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:20.504 INFO:teuthology.orchestra.run.smithi096.stdout:4164 trunc from 0x2a982ff to 0x895fe0 2024-09-22T01:15:20.571 INFO:teuthology.orchestra.run.smithi096.stdout:4165 read 0x2ceafa thru 0x2d7f26 (0x942d bytes) 2024-09-22T01:15:20.571 INFO:teuthology.orchestra.run.smithi096.stdout:4166 punch from 0x1cc47d to 0x1dae33, (0xe9b6 bytes) 2024-09-22T01:15:20.580 INFO:teuthology.orchestra.run.smithi096.stdout:4167 trunc from 0x895fe0 to 0x4531240 2024-09-22T01:15:20.593 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:20.589+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:20.598 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.598 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.599 INFO:teuthology.orchestra.run.smithi096.stdout:4168 write 0x52f5644 thru 0x52ff8ac (0xa269 bytes) 2024-09-22T01:15:20.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:20.630 INFO:teuthology.orchestra.run.smithi096.stdout:4169 write 0xe563aeb thru 0xe572254 (0xe76a bytes) 2024-09-22T01:15:20.638 INFO:teuthology.orchestra.run.smithi096.stdout:4172 punch from 0x4511b08 to 0x451dcd2, (0xc1ca bytes) 2024-09-22T01:15:20.638 INFO:teuthology.orchestra.run.smithi096.stdout:4173 write 0x9db6a5a thru 0x9db6efa (0x4a1 bytes) 2024-09-22T01:15:20.642 INFO:teuthology.orchestra.run.smithi096.stdout:4174 write 0xc168d95 thru 0xc1745e0 (0xb84c bytes) 2024-09-22T01:15:20.662 INFO:teuthology.orchestra.run.smithi096.stdout:4177 trunc from 0xe572255 to 0x805ae0e 2024-09-22T01:15:20.684 INFO:teuthology.orchestra.run.smithi096.stdout:4178 read 0x19cfcf7 thru 0x19d22be (0x25c8 bytes) 2024-09-22T01:15:20.694 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:20.690+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:20.708 INFO:teuthology.orchestra.run.smithi096.stdout:4179 writesame 0xcd7f722 thru 0xcd8ab94 data_size 0x1(0xb473 bytes) 2024-09-22T01:15:20.708 INFO:teuthology.orchestra.run.smithi096.stdout:4180 write 0x14ac12 thru 0x14f5d4 (0x49c3 bytes) 2024-09-22T01:15:20.712 INFO:teuthology.orchestra.run.smithi096.stdout:4181 write 0x6914797 thru 0x691b16b (0x69d5 bytes) 2024-09-22T01:15:20.718 INFO:teuthology.orchestra.run.smithi096.stdout:4183 write 0x230776e thru 0x2308056 (0x8e9 bytes) 2024-09-22T01:15:20.725 INFO:teuthology.orchestra.run.smithi096.stdout:4184 writesame 0xa817d86 thru 0xa82014e data_size 0x1(0x83c9 bytes) 2024-09-22T01:15:20.733 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.733 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.748 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:20.795 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:20.789+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:4185 writesame 0xe22e427 thru 0xe23a2f7 data_size 0x13(0xbed1 bytes) 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:4186 punch from 0x125d155 to 0x1264070, (0x6f1b bytes) 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:4187 read 0x7ecc744 thru 0x7ed99b4 (0xd271 bytes) 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:4189 punch from 0x6cd277c to 0x6cd7516, (0x4d9a bytes) 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:4190 read 0x1dcd8cc thru 0x1dd590e (0x8043 bytes) 2024-09-22T01:15:20.803 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:20.818 INFO:teuthology.orchestra.run.smithi096.stdout:4194 write 0xedab85f thru 0xedb2830 (0x6fd2 bytes) 2024-09-22T01:15:20.824 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:20.824 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:20.825 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-22T01:15:20.825 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:15:20.825 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:15:20.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-22T01:15:20.859 INFO:teuthology.orchestra.run.smithi096.stdout:4195 trunc from 0xedb2831 to 0x19c0720 2024-09-22T01:15:20.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:15:20.896 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:20.890+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:20.910 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:20.910 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:20.914 INFO:teuthology.orchestra.run.smithi096.stdout:4198 writesame 0x4037432 thru 0x4040a1f data_size 0x3(0x95ee bytes) 2024-09-22T01:15:20.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:20.928 INFO:teuthology.orchestra.run.smithi096.stdout:4199 write 0x5d7d61f thru 0x5d88665 (0xb047 bytes) 2024-09-22T01:15:20.934 INFO:teuthology.orchestra.run.smithi096.stdout:4200 punch from 0x4de546a to 0x4de8597, (0x312d bytes) 2024-09-22T01:15:20.937 INFO:teuthology.orchestra.run.smithi096.stdout:4201 trunc from 0x5d88666 to 0x6b6f87f 2024-09-22T01:15:20.944 INFO:teuthology.orchestra.run.smithi096.stdout:4203 read 0x48311bd thru 0x4834f65 (0x3da9 bytes) 2024-09-22T01:15:20.945 INFO:teuthology.orchestra.run.smithi096.stdout:4205 trunc from 0x6b6f87f to 0x6a82d06 2024-09-22T01:15:20.961 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:20.961 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-22T01:15:20.962 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:4206 writesame 0xd8f5794 thru 0xd8ff677 data_size 0x2(0x9ee4 bytes) 2024-09-22T01:15:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:4207 punch from 0x3e1ab17 to 0x3e203ba, (0x58a3 bytes) 2024-09-22T01:15:20.969 INFO:teuthology.orchestra.run.smithi096.stdout:4208 write 0x4717e04 thru 0x47249c1 (0xcbbe bytes) 2024-09-22T01:15:20.973 INFO:teuthology.orchestra.run.smithi096.stdout:4209 writesame 0x336c4f8 thru 0x33753cb data_size 0x42(0x8ed4 bytes) 2024-09-22T01:15:20.978 INFO:teuthology.orchestra.run.smithi096.stdout:4210 read 0x9fc56ff thru 0x9fc97bc (0x40be bytes) 2024-09-22T01:15:20.987 INFO:teuthology.orchestra.run.smithi096.stdout:4211 trunc from 0xd8ff678 to 0x71c9d12 2024-09-22T01:15:20.995 INFO:teuthology.orchestra.run.smithi096.stdout:4212 punch from 0x24edcab to 0x24effb4, (0x2309 bytes) 2024-09-22T01:15:20.995 INFO:teuthology.orchestra.run.smithi096.stdout:4213 write 0x627e86b thru 0x62813b6 (0x2b4c bytes) 2024-09-22T01:15:20.997 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:20.993+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:4215 read 0x66c391c thru 0x66cfb78 (0xc25d bytes) 2024-09-22T01:15:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:4216 read 0x21446f3 thru 0x214f7d4 (0xb0e2 bytes) 2024-09-22T01:15:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:4217 punch from 0x54c4ba to 0x55be55, (0xf99b bytes) 2024-09-22T01:15:21.000 INFO:teuthology.orchestra.run.smithi096.stdout:4218 read 0x13386e1 thru 0x1346147 (0xda67 bytes) 2024-09-22T01:15:21.004 INFO:teuthology.orchestra.run.smithi096.stdout:4219 trunc from 0x71c9d12 to 0x9137361 2024-09-22T01:15:21.014 INFO:teuthology.orchestra.run.smithi096.stdout:4220 trunc from 0x9137361 to 0xb3259d6 2024-09-22T01:15:21.027 INFO:teuthology.orchestra.run.smithi096.stdout:4221 trunc from 0xb3259d6 to 0xe74d9be 2024-09-22T01:15:21.035 INFO:teuthology.orchestra.run.smithi096.stdout:4222 write 0xb6cb9bd thru 0xb6cc85b (0xe9f bytes) 2024-09-22T01:15:21.040 INFO:teuthology.orchestra.run.smithi096.stdout:4225 read 0xcbc7ffc thru 0xcbd0a5a (0x8a5f bytes) 2024-09-22T01:15:21.040 INFO:teuthology.orchestra.run.smithi096.stdout:4226 read 0x3e0d128 thru 0x3e130dc (0x5fb5 bytes) 2024-09-22T01:15:21.041 INFO:teuthology.orchestra.run.smithi096.stdout:4229 read 0x6ba719 thru 0x6c1a4e (0x7336 bytes) 2024-09-22T01:15:21.041 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:21.041 INFO:teuthology.orchestra.run.smithi096.stdout:4233 writesame 0x5a2a5dc thru 0x5a2eee1 data_size 0xd(0x4906 bytes) 2024-09-22T01:15:21.046 INFO:teuthology.orchestra.run.smithi096.stdout:4234 writesame 0x69ae7b3 thru 0x69b20c1 data_size 0x9(0x390f bytes) 2024-09-22T01:15:21.050 INFO:teuthology.orchestra.run.smithi096.stdout:4235 punch from 0x20574da to 0x20621a7, (0xaccd bytes) 2024-09-22T01:15:21.050 INFO:teuthology.orchestra.run.smithi096.stdout:4237 write 0x4c23bc0 thru 0x4c25a55 (0x1e96 bytes) 2024-09-22T01:15:21.061 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:21.061 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:21.074 INFO:teuthology.orchestra.run.smithi096.stdout:4239 trunc from 0xe74d9be to 0x1651eb2 2024-09-22T01:15:21.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:21.114 INFO:teuthology.orchestra.run.smithi096.stdout:4241 write 0x9b03347 thru 0x9b07411 (0x40cb bytes) 2024-09-22T01:15:21.115 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:21.109+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:21.118 INFO:teuthology.orchestra.run.smithi096.stdout:4242 write 0x6772672 thru 0x6775059 (0x29e8 bytes) 2024-09-22T01:15:21.134 INFO:teuthology.orchestra.run.smithi096.stdout:4243 write 0xbb19b89 thru 0xbb24593 (0xaa0b bytes) 2024-09-22T01:15:21.147 INFO:teuthology.orchestra.run.smithi096.stdout:4244 trunc from 0xbb24594 to 0x5c3957e 2024-09-22T01:15:21.161 INFO:teuthology.orchestra.run.smithi096.stdout:4245 read 0x4ede00c thru 0x4edf3ea (0x13df bytes) 2024-09-22T01:15:21.161 INFO:teuthology.orchestra.run.smithi096.stdout:4246 punch from 0x164a270 to 0x165638b, (0xc11b bytes) 2024-09-22T01:15:21.161 INFO:teuthology.orchestra.run.smithi096.stdout:4248 read 0x2930255 thru 0x2933aa9 (0x3855 bytes) 2024-09-22T01:15:21.161 INFO:teuthology.orchestra.run.smithi096.stdout:4249 writesame 0x18dfd3a thru 0x18e49c2 data_size 0x15(0x4c89 bytes) 2024-09-22T01:15:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:4250 write 0xa7a02f2 thru 0xa7abb8f (0xb89e bytes) 2024-09-22T01:15:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:4253 read 0x49ff59e thru 0x4a05108 (0x5b6b bytes) 2024-09-22T01:15:21.195 INFO:teuthology.orchestra.run.smithi096.stdout:4254 writesame 0xc74ddb9 thru 0xc7502af data_size 0x1(0x24f7 bytes) 2024-09-22T01:15:21.195 INFO:teuthology.orchestra.run.smithi096.stdout:4255 punch from 0x28a3984 to 0x28b34be, (0xfb3a bytes) 2024-09-22T01:15:21.195 INFO:teuthology.orchestra.run.smithi096.stdout:4256 write 0xa4e476e thru 0xa4ef76f (0xb002 bytes) 2024-09-22T01:15:21.198 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:21.194+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:21.202 INFO:teuthology.orchestra.run.smithi096.stdout:4257 read 0x3fb8199 thru 0x3fbc8a1 (0x4709 bytes) 2024-09-22T01:15:21.215 INFO:teuthology.orchestra.run.smithi096.stdout:4258 trunc from 0xc7502b0 to 0x314e825 2024-09-22T01:15:21.230 INFO:teuthology.orchestra.run.smithi096.stdout:4259 punch from 0x78ffb9 to 0x79407d, (0x40c4 bytes) 2024-09-22T01:15:21.244 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:21.244 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:21.252 INFO:teuthology.orchestra.run.smithi096.stdout:4260 write 0xbc125e2 thru 0xbc16913 (0x4332 bytes) 2024-09-22T01:15:21.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:21.267 INFO:teuthology.orchestra.run.smithi096.stdout:4261 write 0xe769295 thru 0xe769b86 (0x8f2 bytes) 2024-09-22T01:15:21.271 INFO:teuthology.orchestra.run.smithi096.stdout:4262 read 0x8e4ed1e thru 0x8e573ee (0x86d1 bytes) 2024-09-22T01:15:21.271 INFO:teuthology.orchestra.run.smithi096.stdout:4264 clone 36 order 20 su 262144 sc 10 2024-09-22T01:15:21.299 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:21.298+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:21.382 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:21.383 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:21.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:21.400 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:21.394+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:21.500 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:21.497+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:21.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:21.568 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:21.568 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:21.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:21.602 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:21.602+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:21.695 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:21.696 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:21.702 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:21.697+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:21.713 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:21.713 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:21.729 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:21.743 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:21.743 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:21.743 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:21.746 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:21.758 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:21.768 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:21.768 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:21.789 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:21.789 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:21.803 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:21.802+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:21.809 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:21.809 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:21.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:21.830 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:21.843 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 90.913757249, 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:15:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:15:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57902 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4516864~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.708031820999999, 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055853368, 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57902 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190193+0000", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190194+0000", 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190196+0000", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1100000000000001e-06 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190202+0000", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1519999999999997e-06 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190207+0000", 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8960000000000002e-06 2024-09-22T01:15:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245857+0000", 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055649864 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.245932+0000", 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4938999999999995e-05 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246047+0000", 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115036 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.70800959, 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:15:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707976817999999, 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:15:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707955353999999, 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:15:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707934411, 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707914503000001, 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:15:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707894121999999, 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:15:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707872184999999, 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:15:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:15:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:15:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.707362604, 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:15:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975981557000001, 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:15:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:15:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975947253000001, 2024-09-22T01:15:21.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:15:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975876609, 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:15:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975724905, 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:15:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:15:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975509368000001, 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975296143000001, 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:15:21.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.975098019000001, 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.973 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:15:21.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.977 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.974894227, 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:15:21.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:15:21.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:15:21.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.974643402000002, 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:15:21.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:21.986 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.974424381999999, 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:21.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:21.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:21.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:21.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:21.991 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:21.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:21.992 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone35 from 0xe769b87 (overlap 0x895fe0) to 0x0 2024-09-22T01:15:21.992 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:21.989+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:22.092 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:22.089+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:22.127 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-22T01:15:22.161 INFO:teuthology.orchestra.run.smithi096.stdout:4265 read 0x2df102a thru 0x2dff772 (0xe749 bytes) 2024-09-22T01:15:22.164 INFO:teuthology.orchestra.run.smithi096.stdout:4266 writesame 0x2552a67 thru 0x25621f1 data_size 0xb(0xf78b bytes) 2024-09-22T01:15:22.174 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:22.174 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:22.175 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:22.175 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:22.185 INFO:teuthology.orchestra.run.smithi096.stdout:4267 read 0x5b16571 thru 0x5b22815 (0xc2a5 bytes) 2024-09-22T01:15:22.185 INFO:teuthology.orchestra.run.smithi096.stdout:4268 punch from 0x650db07 to 0x650fdb6, (0x22af bytes) 2024-09-22T01:15:22.185 INFO:teuthology.orchestra.run.smithi096.stdout:4271 write 0x95c5688 thru 0x95cdf70 (0x88e9 bytes) 2024-09-22T01:15:22.190 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:22.194 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:22.194+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:4272 punch from 0xc764e14 to 0xc771432, (0xc61e bytes) 2024-09-22T01:15:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:4274 read 0x19ddb3d thru 0x19ed320 (0xf7e4 bytes) 2024-09-22T01:15:22.197 INFO:teuthology.orchestra.run.smithi096.stdout:4275 write 0x3a9dce1 thru 0x3aa54f9 (0x7819 bytes) 2024-09-22T01:15:22.204 INFO:teuthology.orchestra.run.smithi096.stdout:4276 read 0xb83ab4d thru 0xb8474e0 (0xc994 bytes) 2024-09-22T01:15:22.204 INFO:teuthology.orchestra.run.smithi096.stdout:4277 write 0xa54f801 thru 0xa553c6b (0x446b bytes) 2024-09-22T01:15:22.210 INFO:teuthology.orchestra.run.smithi096.stdout:4278 write 0x7eaaa14 thru 0x7eb4e8a (0xa477 bytes) 2024-09-22T01:15:22.216 INFO:teuthology.orchestra.run.smithi096.stdout:4279 punch from 0x2289895 to 0x229407f, (0xa7ea bytes) 2024-09-22T01:15:22.216 INFO:teuthology.orchestra.run.smithi096.stdout:4280 punch from 0x19ac457 to 0x19ba153, (0xdcfc bytes) 2024-09-22T01:15:22.216 INFO:teuthology.orchestra.run.smithi096.stdout:4281 write 0x94e2f3 thru 0x9525fe (0x430c bytes) 2024-09-22T01:15:22.227 INFO:teuthology.orchestra.run.smithi096.stdout:4283 write 0xc9e5d4a thru 0xc9f0311 (0xa5c8 bytes) 2024-09-22T01:15:22.234 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:22.235 INFO:teuthology.orchestra.run.smithi096.stdout:4285 read 0x623c5c thru 0x633112 (0xf4b7 bytes) 2024-09-22T01:15:22.236 INFO:teuthology.orchestra.run.smithi096.stdout:4287 write 0x26f6d59 thru 0x26ff98f (0x8c37 bytes) 2024-09-22T01:15:22.261 INFO:teuthology.orchestra.run.smithi096.stdout:4288 trunc from 0xe769b87 to 0x3918a13 2024-09-22T01:15:22.294 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:22.289+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:22.331 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:22.331 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:22.331 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:22.331 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:22.332 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:22.335 INFO:teuthology.orchestra.run.smithi096.stdout:4290 write 0xdaf0fc4 thru 0xdaf8dcf (0x7e0c bytes) 2024-09-22T01:15:22.344 INFO:teuthology.orchestra.run.smithi096.stdout:4291 write 0x81beb86 thru 0x81c4399 (0x5814 bytes) 2024-09-22T01:15:22.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:22.351 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:22.359 INFO:teuthology.orchestra.run.smithi096.stdout:4293 trunc from 0xdaf8dd0 to 0x8cb9f4c 2024-09-22T01:15:22.379 INFO:teuthology.orchestra.run.smithi096.stdout:4294 punch from 0x7d0712 to 0x7e0305, (0xfbf3 bytes) 2024-09-22T01:15:22.379 INFO:teuthology.orchestra.run.smithi096.stdout:4296 write 0x2c4e1bc thru 0x2c5463b (0x6480 bytes) 2024-09-22T01:15:22.386 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:22.386 INFO:teuthology.orchestra.run.smithi096.stdout:4298 writesame 0x7f1c084 thru 0x7f1c68d data_size 0x1(0x60a bytes) 2024-09-22T01:15:22.389 INFO:teuthology.orchestra.run.smithi096.stdout:4299 read 0x2aab8b1 thru 0x2ab7e87 (0xc5d7 bytes) 2024-09-22T01:15:22.389 INFO:teuthology.orchestra.run.smithi096.stdout:4300 read 0x3fb7796 thru 0x3fbe6f7 (0x6f62 bytes) 2024-09-22T01:15:22.395 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:22.390+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:22.399 INFO:teuthology.orchestra.run.smithi096.stdout:4305 write 0x98de8fc thru 0x98e4dd7 (0x64dc bytes) 2024-09-22T01:15:22.406 INFO:teuthology.orchestra.run.smithi096.stdout:4306 punch from 0x16bf521 to 0x16c7fad, (0x8a8c bytes) 2024-09-22T01:15:22.406 INFO:teuthology.orchestra.run.smithi096.stdout:4307 clone 37 order 18 su 16384 sc 4 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.575290707000001, 2024-09-22T01:15:22.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:15:22.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.575126351000002, 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:15:22.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.484 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570434575, 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:15:22.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:15:22.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570397021000002, 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:15:22.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:15:22.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:22.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570364859000001, 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:22.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570342522000001, 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:15:22.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:22.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570321161999999, 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:15:22.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.570299972000001, 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:15:22.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566394479, 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:15:22.511 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566372298000001, 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.516 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:22.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566351019999999, 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:15:22.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:22.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566328673000001, 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:22.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566306997000002, 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:15:22.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.566285574999998, 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:15:22.533 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:15:22.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.559646829999998, 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.538 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:15:22.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.559430082999999, 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:22.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:15:22.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:15:22.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.559047456000002, 2024-09-22T01:15:22.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:22.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.550 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.558641915999999, 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.555425454999998, 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:15:22.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.518640059999999, 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:22.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:22.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:22.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:22.564 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:22.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:22.566 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:22.561+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:22.666 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:22.661+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:22.733 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:22.733 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:22.733 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:22.733 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:22.747 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:22.792 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:22.789+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:22.868 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:22.865+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:22.917 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:22.917 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:22.934 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:22.969 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:22.966+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:23.004 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone36 from 0x98e4dd8 (overlap 0x3918a13) to 0x5b5a9b 2024-09-22T01:15:23.070 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:23.066+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.110112400999995, 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.562548605000003, 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 43.55123794, 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.312886855999999, 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.983011382000001, 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.983000361, 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982989537, 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982978658, 2024-09-22T01:15:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982964644999999, 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.98293372, 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982908632000001, 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982884396999999, 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982862381, 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:23.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:23.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:23.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982832871999999, 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:23.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982781187000001, 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982748558000001, 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.98269118, 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:23.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982676116, 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:23.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982663192, 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:23.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.982651593, 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:23.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:23.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:23.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:23.224 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:23.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:23.226 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-22T01:15:23.226 INFO:teuthology.orchestra.run.smithi096.stdout:4308 punch from 0x59e75fa to 0x59e94a2, (0x1ea8 bytes) 2024-09-22T01:15:23.227 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:23.226+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:23.235 INFO:teuthology.orchestra.run.smithi096.stdout:4309 write 0xad7129c thru 0xad74d19 (0x3a7e bytes) 2024-09-22T01:15:23.241 INFO:teuthology.orchestra.run.smithi096.stdout:4310 write 0x3556c79 thru 0x355bdc2 (0x514a bytes) 2024-09-22T01:15:23.250 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.250 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.250 INFO:teuthology.orchestra.run.smithi096.stdout:4316 read 0x7121862 thru 0x712b3ab (0x9b4a bytes) 2024-09-22T01:15:23.253 INFO:teuthology.orchestra.run.smithi096.stdout:4318 trunc from 0xad74d1a to 0x93eccaf 2024-09-22T01:15:23.288 INFO:teuthology.orchestra.run.smithi096.stdout:4319 trunc from 0x93eccaf to 0x6568ad0 2024-09-22T01:15:23.326 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:23.321+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:23.337 INFO:teuthology.orchestra.run.smithi096.stdout:4320 trunc from 0x6568ad0 to 0xed3f6fe 2024-09-22T01:15:23.347 INFO:teuthology.orchestra.run.smithi096.stdout:4321 punch from 0x3c4b492 to 0x3c4c4d2, (0x1040 bytes) 2024-09-22T01:15:23.347 INFO:teuthology.orchestra.run.smithi096.stdout:4322 write 0x6c33d59 thru 0x6c43888 (0xfb30 bytes) 2024-09-22T01:15:23.357 INFO:teuthology.orchestra.run.smithi096.stdout:4323 punch from 0x7b76e4 to 0x7b853d, (0xe59 bytes) 2024-09-22T01:15:23.358 INFO:teuthology.orchestra.run.smithi096.stdout:4324 write 0x72be255 thru 0x72cca8f (0xe83b bytes) 2024-09-22T01:15:23.360 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:23.361 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:23.361 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:23.361 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:23.376 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:23.390 INFO:teuthology.orchestra.run.smithi096.stdout:4325 trunc from 0xed3f6fe to 0x5f35c59 2024-09-22T01:15:23.427 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.428 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:23.425+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:23.447 INFO:teuthology.orchestra.run.smithi096.stdout:4327 write 0x941d101 thru 0x94264a0 (0x93a0 bytes) 2024-09-22T01:15:23.456 INFO:teuthology.orchestra.run.smithi096.stdout:4328 writesame 0x55f1963 thru 0x55f4c04 data_size 0x1(0x32a2 bytes) 2024-09-22T01:15:23.475 INFO:teuthology.orchestra.run.smithi096.stdout:4330 write 0x423c961 thru 0x4241e57 (0x54f7 bytes) 2024-09-22T01:15:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:4332 punch from 0xec46ba to 0xec7122, (0x2a68 bytes) 2024-09-22T01:15:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:4334 read 0x759e665 thru 0x759e8b5 (0x251 bytes) 2024-09-22T01:15:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:4335 write 0x58fdbc0 thru 0x59097bd (0xbbfe bytes) 2024-09-22T01:15:23.504 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:23.504 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:23.504 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:23.504 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:23.504 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:23.516 INFO:teuthology.orchestra.run.smithi096.stdout:4336 trunc from 0x94264a1 to 0x272cfea 2024-09-22T01:15:23.519 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:23.529 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:23.525+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:23.595 INFO:teuthology.orchestra.run.smithi096.stdout:4337 writesame 0xa7697e1 thru 0xa771786 data_size 0x1(0x7fa6 bytes) 2024-09-22T01:15:23.617 INFO:teuthology.orchestra.run.smithi096.stdout:4338 write 0xe9dbdf8 thru 0xe9e48a4 (0x8aad bytes) 2024-09-22T01:15:23.629 INFO:teuthology.orchestra.run.smithi096.stdout:4340 write 0x26ebefd thru 0x26fa8a0 (0xe9a4 bytes) 2024-09-22T01:15:23.630 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:23.630+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:23.650 INFO:teuthology.orchestra.run.smithi096.stdout:4341 writesame 0x9f8bfba thru 0x9f9192c data_size 0x3(0x5973 bytes) 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296792941, 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:23.652 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:23.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296762671000003, 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:23.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296733097999997, 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.661 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:23.665 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296679007000002, 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:23.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.669 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296623163, 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.670 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:23.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:23.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296565272999999, 2024-09-22T01:15:23.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:23.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296503540000003, 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:23.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:23.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296408077000002, 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.683 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:23.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296346253999999, 2024-09-22T01:15:23.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:23.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:23.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296255113999997, 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296193207999998, 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.696 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:23.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296132720999999, 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.700 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:23.704 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296075250999998, 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:23.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:23.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.296017970999998, 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:23.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:23.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.295958446999997, 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.713 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:23.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:23.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:23.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.295905562999998, 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:23.717 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:23.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.286793000999999, 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:23.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.843533815000001, 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:23.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:23.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.822462035000001, 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:23.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:23.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.636144476, 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:23.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:23.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.741 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:23.743 INFO:teuthology.orchestra.run.smithi096.stdout:4342 write 0x1b8f7e7 thru 0x1b9ef77 (0xf791 bytes) 2024-09-22T01:15:23.743 INFO:teuthology.orchestra.run.smithi096.stdout:4343 read 0x176d99a thru 0x177b471 (0xdad8 bytes) 2024-09-22T01:15:23.743 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.743 INFO:teuthology.orchestra.run.smithi096.stdout:4345 write 0x63c3b9 thru 0x64630c (0x9f54 bytes) 2024-09-22T01:15:23.743 INFO:teuthology.orchestra.run.smithi096.stdout:4346 trunc from 0xe9e48a5 to 0x558abe7 2024-09-22T01:15:23.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:23.744 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:23.742+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:23.758 INFO:teuthology.orchestra.run.smithi096.stdout:4347 punch from 0x22e0069 to 0x22ee92a, (0xe8c1 bytes) 2024-09-22T01:15:23.758 INFO:teuthology.orchestra.run.smithi096.stdout:4349 read 0x4e5d987 thru 0x4e68365 (0xa9df bytes) 2024-09-22T01:15:23.758 INFO:teuthology.orchestra.run.smithi096.stdout:4350 read 0x3a096b9 thru 0x3a0f9d9 (0x6321 bytes) 2024-09-22T01:15:23.759 INFO:teuthology.orchestra.run.smithi096.stdout:4351 read 0x1698c64 thru 0x169fae9 (0x6e86 bytes) 2024-09-22T01:15:23.760 INFO:teuthology.orchestra.run.smithi096.stdout:4352 read 0x67b733 thru 0x681abd (0x638b bytes) 2024-09-22T01:15:23.761 INFO:teuthology.orchestra.run.smithi096.stdout:4353 writesame 0x35c0a33 thru 0x35cb931 data_size 0x6d(0xaeff bytes) 2024-09-22T01:15:23.796 INFO:teuthology.orchestra.run.smithi096.stdout:4354 writesame 0x810e1dc thru 0x8113d30 data_size 0x1(0x5b55 bytes) 2024-09-22T01:15:23.819 INFO:teuthology.orchestra.run.smithi096.stdout:4356 write 0xbdbd14b thru 0xbdc13be (0x4274 bytes) 2024-09-22T01:15:23.827 INFO:teuthology.orchestra.run.smithi096.stdout:4358 writesame 0x9768908 thru 0x9769c77 data_size 0x8(0x1370 bytes) 2024-09-22T01:15:23.835 INFO:teuthology.orchestra.run.smithi096.stdout:4360 read 0x7b9991e thru 0x7b9ed9e (0x5481 bytes) 2024-09-22T01:15:23.836 INFO:teuthology.orchestra.run.smithi096.stdout:4363 read 0xf7a837 thru 0xf7e3b6 (0x3b80 bytes) 2024-09-22T01:15:23.837 INFO:teuthology.orchestra.run.smithi096.stdout:4364 write 0x1800279 thru 0x18012ae (0x1036 bytes) 2024-09-22T01:15:23.843 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:23.842+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:23.848 INFO:teuthology.orchestra.run.smithi096.stdout:4365 read 0xd8b454 thru 0xd8c8e1 (0x148e bytes) 2024-09-22T01:15:23.850 INFO:teuthology.orchestra.run.smithi096.stdout:4366 punch from 0x5da6437 to 0x5da83ed, (0x1fb6 bytes) 2024-09-22T01:15:23.851 INFO:teuthology.orchestra.run.smithi096.stdout:4368 punch from 0x2b569b4 to 0x2b581b4, (0x1800 bytes) 2024-09-22T01:15:23.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:23.854 INFO:teuthology.orchestra.run.smithi096.stdout:4369 trunc from 0xbdc13bf to 0xcf7b2a2 2024-09-22T01:15:23.866 INFO:teuthology.orchestra.run.smithi096.stdout:4370 writesame 0x60ece94 thru 0x60f91ac data_size 0x7(0xc319 bytes) 2024-09-22T01:15:23.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:15:23.883 INFO:teuthology.orchestra.run.smithi096.stdout:4373 trunc from 0xcf7b2a2 to 0xbcb423f 2024-09-22T01:15:23.900 INFO:teuthology.orchestra.run.smithi096.stdout:4374 write 0x80d988d thru 0x80e9567 (0xfcdb bytes) 2024-09-22T01:15:23.910 INFO:teuthology.orchestra.run.smithi096.stdout:4378 read 0x5c9def4 thru 0x5cac166 (0xe273 bytes) 2024-09-22T01:15:23.910 INFO:teuthology.orchestra.run.smithi096.stdout:4379 write 0x4b57863 thru 0x4b5f5c9 (0x7d67 bytes) 2024-09-22T01:15:23.917 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.918 INFO:teuthology.orchestra.run.smithi096.stdout:4383 write 0x242ea32 thru 0x2431b05 (0x30d4 bytes) 2024-09-22T01:15:23.925 INFO:teuthology.orchestra.run.smithi096.stdout:4384 read 0x8696ca7 thru 0x86989b1 (0x1d0b bytes) 2024-09-22T01:15:23.925 INFO:teuthology.orchestra.run.smithi096.stdout:4385 write 0x410a239 thru 0x410df1d (0x3ce5 bytes) 2024-09-22T01:15:23.930 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:23.930 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:23.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.930 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:112310 3.4 e88/24 3:23f91fce:::rbd_data.11243103bbf7.0000000000000107:head v 88'266, mlcod=88'264)", 2024-09-22T01:15:23.930 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:23.930560+0000", 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 0.00024499400000000001, 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000275667, 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930560+0000", 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930560+0000", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930562+0000", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.593e-06 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930582+0000", 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0186e-05 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930609+0000", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6803000000000001e-05 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930622+0000", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2666e-05 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930744+0000", 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012228299999999999 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:23.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:23.930803+0000", 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9469000000000003e-05 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: ], 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 1 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:23.934 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.935 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.935 INFO:teuthology.orchestra.run.smithi096.stdout:4389 write 0x360a37d thru 0x360ca6d (0x26f1 bytes) 2024-09-22T01:15:23.939 INFO:teuthology.orchestra.run.smithi096.stdout:4390 write 0xbb85c14 thru 0xbb94600 (0xe9ed bytes) 2024-09-22T01:15:23.944 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:23.942+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:23.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:23.947 INFO:teuthology.orchestra.run.smithi096.stdout:4392 write 0x2a686d3 thru 0x2a75e22 (0xd750 bytes) 2024-09-22T01:15:23.956 INFO:teuthology.orchestra.run.smithi096.stdout:4394 read 0x34df2d6 thru 0x34eb629 (0xc354 bytes) 2024-09-22T01:15:23.956 INFO:teuthology.orchestra.run.smithi096.stdout:4395 punch from 0x128c579 to 0x129817c, (0xbc03 bytes) 2024-09-22T01:15:23.956 INFO:teuthology.orchestra.run.smithi096.stdout:4396 punch from 0xc8893 to 0xce57c, (0x5ce9 bytes) 2024-09-22T01:15:23.956 INFO:teuthology.orchestra.run.smithi096.stdout:4397 punch from 0x6b44ece to 0x6b481db, (0x330d bytes) 2024-09-22T01:15:23.956 INFO:teuthology.orchestra.run.smithi096.stdout:4400 write 0x5556786 thru 0x5565e42 (0xf6bd bytes) 2024-09-22T01:15:23.964 INFO:teuthology.orchestra.run.smithi096.stdout:4401 punch from 0x42d3a0a to 0x42d908c, (0x5682 bytes) 2024-09-22T01:15:23.964 INFO:teuthology.orchestra.run.smithi096.stdout:4402 punch from 0x873ca5a to 0x873e49d, (0x1a43 bytes) 2024-09-22T01:15:23.964 INFO:teuthology.orchestra.run.smithi096.stdout:4403 read 0x2882f97 thru 0x2892a48 (0xfab2 bytes) 2024-09-22T01:15:23.964 INFO:teuthology.orchestra.run.smithi096.stdout:4404 write 0xa4b302a thru 0xa4b6b15 (0x3aec bytes) 2024-09-22T01:15:23.970 INFO:teuthology.orchestra.run.smithi096.stdout:4405 read 0x376623c thru 0x3774006 (0xddcb bytes) 2024-09-22T01:15:23.976 INFO:teuthology.orchestra.run.smithi096.stdout:4406 trunc from 0xbcb423f to 0x7ff6ee7 2024-09-22T01:15:23.993 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:23.993 INFO:teuthology.orchestra.run.smithi096.stdout:4410 punch from 0x38c6f61 to 0x38d501d, (0xe0bc bytes) 2024-09-22T01:15:24.006 INFO:teuthology.orchestra.run.smithi096.stdout:4411 trunc from 0x7ff6ee7 to 0x56aaef 2024-09-22T01:15:24.033 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:24.033 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:24.045 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:24.042+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:24.056 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:24.056 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:24.070 INFO:teuthology.orchestra.run.smithi096.stdout:4412 write 0xaff38fe thru 0xafff665 (0xbd68 bytes) 2024-09-22T01:15:24.077 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:24.077 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:24.080 INFO:teuthology.orchestra.run.smithi096.stdout:4413 write 0x2790355 thru 0x2790619 (0x2c5 bytes) 2024-09-22T01:15:24.084 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.084 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.085 INFO:teuthology.orchestra.run.smithi096.stdout:4420 punch from 0x55789b4 to 0x558691f, (0xdf6b bytes) 2024-09-22T01:15:24.085 INFO:teuthology.orchestra.run.smithi096.stdout:4421 read 0x374a04a thru 0x3754bd0 (0xab87 bytes) 2024-09-22T01:15:24.085 INFO:teuthology.orchestra.run.smithi096.stdout:4423 read 0x9b60540 thru 0x9b63e0e (0x38cf bytes) 2024-09-22T01:15:24.085 INFO:teuthology.orchestra.run.smithi096.stdout:4424 punch from 0xa305092 to 0xa30aaf1, (0x5a5f bytes) 2024-09-22T01:15:24.086 INFO:teuthology.orchestra.run.smithi096.stdout:4425 trunc from 0xafff666 to 0x9cc8539 2024-09-22T01:15:24.097 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:24.097 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:24.103 INFO:teuthology.orchestra.run.smithi096.stdout:4426 read 0x8cf88c3 thru 0x8cfd671 (0x4daf bytes) 2024-09-22T01:15:24.103 INFO:teuthology.orchestra.run.smithi096.stdout:4428 read 0x14690d3 thru 0x1479030 (0xff5e bytes) 2024-09-22T01:15:24.103 INFO:teuthology.orchestra.run.smithi096.stdout:4430 writesame 0x1b4b68f thru 0x1b539e9 data_size 0xb(0x835b bytes) 2024-09-22T01:15:24.112 INFO:teuthology.orchestra.run.smithi096.stdout:4431 writesame 0x6cd1d06 thru 0x6cde63f data_size 0x2b(0xc93a bytes) 2024-09-22T01:15:24.119 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:24.120 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:24.120 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:24.120 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:24.120 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:24.122 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:24.125 INFO:teuthology.orchestra.run.smithi096.stdout:4432 write 0x890e008 thru 0x891638c (0x8385 bytes) 2024-09-22T01:15:24.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:24.142 INFO:teuthology.orchestra.run.smithi096.stdout:4433 trunc from 0x9cc8539 to 0x50e1b25 2024-09-22T01:15:24.143 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:24.143 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:24.164 INFO:teuthology.orchestra.run.smithi096.stdout:4434 trunc from 0x50e1b25 to 0x13eb29c 2024-09-22T01:15:24.169 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:24.179 INFO:teuthology.orchestra.run.smithi096.stdout:4436 read 0x112665 thru 0x116f78 (0x4914 bytes) 2024-09-22T01:15:24.182 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:24.182+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:24.189 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:24.189 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:24.194 INFO:teuthology.orchestra.run.smithi096.stdout:4438 write 0x5cfe0e5 thru 0x5d0461a (0x6536 bytes) 2024-09-22T01:15:24.204 INFO:teuthology.orchestra.run.smithi096.stdout:4439 punch from 0x66bad3 to 0x67943c, (0xd969 bytes) 2024-09-22T01:15:24.204 INFO:teuthology.orchestra.run.smithi096.stdout:4441 trunc from 0x5d0461b to 0x5c97a21 2024-09-22T01:15:24.226 INFO:teuthology.orchestra.run.smithi096.stdout:4444 write 0x890cb52 thru 0x8912a12 (0x5ec1 bytes) 2024-09-22T01:15:24.234 INFO:teuthology.orchestra.run.smithi096.stdout:4446 punch from 0x832f93e to 0x833ddfa, (0xe4bc bytes) 2024-09-22T01:15:24.234 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.234 INFO:teuthology.orchestra.run.smithi096.stdout:4449 write 0x4578a41 thru 0x457a43c (0x19fc bytes) 2024-09-22T01:15:24.239 INFO:teuthology.orchestra.run.smithi096.stdout:4450 writesame 0x1397490 thru 0x1399e36 data_size 0x1(0x29a7 bytes) 2024-09-22T01:15:24.245 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:24.241+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:24.247 INFO:teuthology.orchestra.run.smithi096.stdout:4451 writesame 0x187a7b1 thru 0x187d1f7 data_size 0x1(0x2a47 bytes) 2024-09-22T01:15:24.256 INFO:teuthology.orchestra.run.smithi096.stdout:4452 write 0x4b9f68d thru 0x4ba0211 (0xb85 bytes) 2024-09-22T01:15:24.263 INFO:teuthology.orchestra.run.smithi096.stdout:4453 trunc from 0x8912a13 to 0x7811953 2024-09-22T01:15:24.286 INFO:teuthology.orchestra.run.smithi096.stdout:4454 trunc from 0x7811953 to 0xa51ba34 2024-09-22T01:15:24.293 INFO:teuthology.orchestra.run.smithi096.stdout:4455 punch from 0xddefc6 to 0xde81d7, (0x9211 bytes) 2024-09-22T01:15:24.293 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.294 INFO:teuthology.orchestra.run.smithi096.stdout:4457 read 0x58ac58d thru 0x58aee42 (0x28b6 bytes) 2024-09-22T01:15:24.294 INFO:teuthology.orchestra.run.smithi096.stdout:4459 read 0x930a37d thru 0x93131bb (0x8e3f bytes) 2024-09-22T01:15:24.294 INFO:teuthology.orchestra.run.smithi096.stdout:4460 writesame 0x6dde914 thru 0x6de35fd data_size 0x2(0x4cea bytes) 2024-09-22T01:15:24.307 INFO:teuthology.orchestra.run.smithi096.stdout:4461 trunc from 0xa51ba34 to 0xbc11ecf 2024-09-22T01:15:24.312 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:24.312 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:24.314 INFO:teuthology.orchestra.run.smithi096.stdout:4463 punch from 0x94e57a9 to 0x94e70ed, (0x1944 bytes) 2024-09-22T01:15:24.326 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:24.327 INFO:teuthology.orchestra.run.smithi096.stdout:4464 writesame 0xd790cc4 thru 0xd7930d4 data_size 0x1(0x2411 bytes) 2024-09-22T01:15:24.342 INFO:teuthology.orchestra.run.smithi096.stdout:4466 trunc from 0xd7930d5 to 0x5a5f321 2024-09-22T01:15:24.362 INFO:teuthology.orchestra.run.smithi096.stdout:4467 read 0x50ff1d0 thru 0x510492a (0x575b bytes) 2024-09-22T01:15:24.371 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:24.370+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:24.375 INFO:teuthology.orchestra.run.smithi096.stdout:4468 trunc from 0x5a5f321 to 0x917f95 2024-09-22T01:15:24.400 INFO:teuthology.orchestra.run.smithi096.stdout:4469 punch from 0x2ed9b4 to 0x2f72bc, (0x9908 bytes) 2024-09-22T01:15:24.400 INFO:teuthology.orchestra.run.smithi096.stdout:4472 punch from 0x884c40 to 0x891b44, (0xcf04 bytes) 2024-09-22T01:15:24.428 INFO:teuthology.orchestra.run.smithi096.stdout:4473 write 0x5d25bd3 thru 0x5d27587 (0x19b5 bytes) 2024-09-22T01:15:24.448 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:24.445+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:24.453 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:24.453 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:24.468 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:24.470 INFO:teuthology.orchestra.run.smithi096.stdout:4474 writesame 0xe24e236 thru 0xe254c0d data_size 0xc(0x69d8 bytes) 2024-09-22T01:15:24.470 INFO:teuthology.orchestra.run.smithi096.stdout:4475 write 0x2c005f1 thru 0x2c08d62 (0x8772 bytes) 2024-09-22T01:15:24.483 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.483 INFO:teuthology.orchestra.run.smithi096.stdout:4478 write 0x70430db thru 0x705139c (0xe2c2 bytes) 2024-09-22T01:15:24.496 INFO:teuthology.orchestra.run.smithi096.stdout:4479 write 0xc514bc1 thru 0xc519200 (0x4640 bytes) 2024-09-22T01:15:24.506 INFO:teuthology.orchestra.run.smithi096.stdout:4480 writesame 0x69c1bff thru 0x69d165f data_size 0x1(0xfa61 bytes) 2024-09-22T01:15:24.548 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:24.545+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:24.631 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:24.631 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:24.638 INFO:teuthology.orchestra.run.smithi096.stdout:4481 trunc from 0xe254c0e to 0x52c136a 2024-09-22T01:15:24.645 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:24.649 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:24.645+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:24.677 INFO:teuthology.orchestra.run.smithi096.stdout:4482 punch from 0x68f0b7 to 0x68fbf9, (0xb42 bytes) 2024-09-22T01:15:24.677 INFO:teuthology.orchestra.run.smithi096.stdout:4483 read 0xba17c4 thru 0xba9a79 (0x82b6 bytes) 2024-09-22T01:15:24.690 INFO:teuthology.orchestra.run.smithi096.stdout:4484 write 0x57d27e0 thru 0x57dfee5 (0xd706 bytes) 2024-09-22T01:15:24.703 INFO:teuthology.orchestra.run.smithi096.stdout:4486 write 0x36e2c38 thru 0x36f23c9 (0xf792 bytes) 2024-09-22T01:15:24.750 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:24.745+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:24.753 INFO:teuthology.orchestra.run.smithi096.stdout:4487 write 0xea2a67d thru 0xea2f45b (0x4ddf bytes) 2024-09-22T01:15:24.761 INFO:teuthology.orchestra.run.smithi096.stdout:4488 write 0xd71e0cc thru 0xd72b1a6 (0xd0db bytes) 2024-09-22T01:15:24.771 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:24.771 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:24.780 INFO:teuthology.orchestra.run.smithi096.stdout:4489 read 0xa08da4e thru 0xa090dbc (0x336f bytes) 2024-09-22T01:15:24.787 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:24.806 INFO:teuthology.orchestra.run.smithi096.stdout:4490 trunc from 0xea2f45c to 0x507e100 2024-09-22T01:15:24.834 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.851 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:24.845+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:24.851 INFO:teuthology.orchestra.run.smithi096.stdout:4493 trunc from 0x507e100 to 0xb51c861 2024-09-22T01:15:24.853 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:15:24.878 INFO:teuthology.orchestra.run.smithi096.stdout:4494 trunc from 0xb51c861 to 0x86ebf37 2024-09-22T01:15:24.886 INFO:teuthology.orchestra.run.smithi096.stdout:4496 punch from 0x3aa9290 to 0x3ab6fcf, (0xdd3f bytes) 2024-09-22T01:15:24.887 INFO:teuthology.orchestra.run.smithi096.stdout:4497 punch from 0x389f237 to 0x38ac726, (0xd4ef bytes) 2024-09-22T01:15:24.901 INFO:teuthology.orchestra.run.smithi096.stdout:4498 trunc from 0x86ebf37 to 0xd40801 2024-09-22T01:15:24.917 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:24.917 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:24.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:24.959 INFO:teuthology.orchestra.run.smithi096.stdout:4499 write 0xaaf7f3c thru 0xab01d90 (0x9e55 bytes) 2024-09-22T01:15:24.971 INFO:teuthology.orchestra.run.smithi096.stdout:4500 punch from 0x73652f6 to 0x736f299, (0x9fa3 bytes) 2024-09-22T01:15:24.971 INFO:teuthology.orchestra.run.smithi096.stdout:4501 write 0x6c345f3 thru 0x6c38190 (0x3b9e bytes) 2024-09-22T01:15:24.977 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:24.978+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:24.980 INFO:teuthology.orchestra.run.smithi096.stdout:4503 write 0x80b7145 thru 0x80bb080 (0x3f3c bytes) 2024-09-22T01:15:24.990 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:24.990 INFO:teuthology.orchestra.run.smithi096.stdout:4506 write 0x3c5dff9 thru 0x3c665ea (0x85f2 bytes) 2024-09-22T01:15:25.004 INFO:teuthology.orchestra.run.smithi096.stdout:4507 read 0x56e2bb0 thru 0x56eef79 (0xc3ca bytes) 2024-09-22T01:15:25.005 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.005 INFO:teuthology.orchestra.run.smithi096.stdout:4509 read 0x65ef3f9 thru 0x65f2970 (0x3578 bytes) 2024-09-22T01:15:25.005 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.005 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.008 INFO:teuthology.orchestra.run.smithi096.stdout:4512 trunc from 0xab01d91 to 0x96ad68f 2024-09-22T01:15:25.030 INFO:teuthology.orchestra.run.smithi096.stdout:4514 writesame 0x298452a thru 0x2987558 data_size 0x1(0x302f bytes) 2024-09-22T01:15:25.052 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:25.049+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:25.061 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.061 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.070 INFO:teuthology.orchestra.run.smithi096.stdout:4515 trunc from 0x96ad68f to 0x1849d95 2024-09-22T01:15:25.076 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:25.099 INFO:teuthology.orchestra.run.smithi096.stdout:4516 flatten 2024-09-22T01:15:25.153 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:25.149+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:25.165 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.165 INFO:teuthology.orchestra.run.smithi096.stdout:4518 read 0x48ccae thru 0x491974 (0x4cc7 bytes) 2024-09-22T01:15:25.203 INFO:teuthology.orchestra.run.smithi096.stdout:4519 write 0xdec395c thru 0xdecd38d (0x9a32 bytes) 2024-09-22T01:15:25.219 INFO:teuthology.orchestra.run.smithi096.stdout:4522 writesame 0x40d96bc thru 0x40e2043 data_size 0x48(0x8988 bytes) 2024-09-22T01:15:25.245 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.245 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.255 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:25.254+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:25.259 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:25.260 INFO:teuthology.orchestra.run.smithi096.stdout:4523 trunc from 0xdecd38e to 0x27931cf 2024-09-22T01:15:25.281 INFO:teuthology.orchestra.run.smithi096.stdout:4524 writesame 0x1e658ec thru 0x1e7311e data_size 0x13(0xd833 bytes) 2024-09-22T01:15:25.295 INFO:teuthology.orchestra.run.smithi096.stdout:4525 trunc from 0x27931cf to 0x2f30c3f 2024-09-22T01:15:25.302 INFO:teuthology.orchestra.run.smithi096.stdout:4526 read 0x6041ba thru 0x607ad4 (0x391b bytes) 2024-09-22T01:15:25.324 INFO:teuthology.orchestra.run.smithi096.stdout:4527 write 0xae918b7 thru 0xae95b5c (0x42a6 bytes) 2024-09-22T01:15:25.347 INFO:teuthology.orchestra.run.smithi096.stdout:4528 write 0xe5df85b thru 0xe5e0816 (0xfbc bytes) 2024-09-22T01:15:25.355 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:25.349+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:25.356 INFO:teuthology.orchestra.run.smithi096.stdout:4530 read 0x3e7eea6 thru 0x3e8ddf2 (0xef4d bytes) 2024-09-22T01:15:25.356 INFO:teuthology.orchestra.run.smithi096.stdout:4532 read 0xca94584 thru 0xca951fb (0xc78 bytes) 2024-09-22T01:15:25.357 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.357 INFO:teuthology.orchestra.run.smithi096.stdout:4534 write 0x424f04b thru 0x425668c (0x7642 bytes) 2024-09-22T01:15:25.368 INFO:teuthology.orchestra.run.smithi096.stdout:4536 punch from 0x6756017 to 0x675cc34, (0x6c1d bytes) 2024-09-22T01:15:25.368 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.368 INFO:teuthology.orchestra.run.smithi096.stdout:4539 write 0x59d1a4c thru 0x59df664 (0xdc19 bytes) 2024-09-22T01:15:25.383 INFO:teuthology.orchestra.run.smithi096.stdout:4540 read 0x33c68ce thru 0x33cff72 (0x96a5 bytes) 2024-09-22T01:15:25.383 INFO:teuthology.orchestra.run.smithi096.stdout:4542 punch from 0x834769b to 0x8355e73, (0xe7d8 bytes) 2024-09-22T01:15:25.383 INFO:teuthology.orchestra.run.smithi096.stdout:4543 write 0xb3f9ff1 thru 0xb404ac8 (0xaad8 bytes) 2024-09-22T01:15:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:4547 punch from 0xd57f30b to 0xd585592, (0x6287 bytes) 2024-09-22T01:15:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.394 INFO:teuthology.orchestra.run.smithi096.stdout:4552 write 0x5b9061b thru 0x5b91cf6 (0x16dc bytes) 2024-09-22T01:15:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:4553 punch from 0xc982cba to 0xc992267, (0xf5ad bytes) 2024-09-22T01:15:25.400 INFO:teuthology.orchestra.run.smithi096.stdout:4554 write 0xbddcb6 thru 0xbe89cb (0xad16 bytes) 2024-09-22T01:15:25.404 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.404 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.412 INFO:teuthology.orchestra.run.smithi096.stdout:4555 read 0xc7fda17 thru 0xc7ff5cd (0x1bb7 bytes) 2024-09-22T01:15:25.412 INFO:teuthology.orchestra.run.smithi096.stdout:4556 read 0x113762e thru 0x1142ae6 (0xb4b9 bytes) 2024-09-22T01:15:25.413 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.413 INFO:teuthology.orchestra.run.smithi096.stdout:4558 read 0x8792f9d thru 0x8794a7d (0x1ae1 bytes) 2024-09-22T01:15:25.413 INFO:teuthology.orchestra.run.smithi096.stdout:4560 writesame 0xb6db1e thru 0xb7a3d6 data_size 0x5(0xc8b9 bytes) 2024-09-22T01:15:25.421 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:25.427 INFO:teuthology.orchestra.run.smithi096.stdout:4561 punch from 0xc43849b to 0xc43aaad, (0x2612 bytes) 2024-09-22T01:15:25.427 INFO:teuthology.orchestra.run.smithi096.stdout:4562 punch from 0x408b616 to 0x408cb5a, (0x1544 bytes) 2024-09-22T01:15:25.427 INFO:teuthology.orchestra.run.smithi096.stdout:4564 write 0x251dd70 thru 0x25234a4 (0x5735 bytes) 2024-09-22T01:15:25.435 INFO:teuthology.orchestra.run.smithi096.stdout:4565 writesame 0xc3ea0d thru 0xc3fe4d data_size 0x11(0x1441 bytes) 2024-09-22T01:15:25.441 INFO:teuthology.orchestra.run.smithi096.stdout:4569 write 0x788e8eb thru 0x78962dd (0x79f3 bytes) 2024-09-22T01:15:25.451 INFO:teuthology.orchestra.run.smithi096.stdout:4570 punch from 0x2549958 to 0x25577ce, (0xde76 bytes) 2024-09-22T01:15:25.451 INFO:teuthology.orchestra.run.smithi096.stdout:4571 read 0x8b817b thru 0x8c7ae4 (0xf96a bytes) 2024-09-22T01:15:25.451 INFO:teuthology.orchestra.run.smithi096.stdout:4572 write 0x4ac3ec7 thru 0x4ac7b99 (0x3cd3 bytes) 2024-09-22T01:15:25.456 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:25.454+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:25.460 INFO:teuthology.orchestra.run.smithi096.stdout:4573 read 0xd319491 thru 0xd3199ac (0x51c bytes) 2024-09-22T01:15:25.460 INFO:teuthology.orchestra.run.smithi096.stdout:4576 write 0x7135745 thru 0x7144e43 (0xf6ff bytes) 2024-09-22T01:15:25.473 INFO:teuthology.orchestra.run.smithi096.stdout:4577 writesame 0x25e2867 thru 0x25e6508 data_size 0x27(0x3ca2 bytes) 2024-09-22T01:15:25.480 INFO:teuthology.orchestra.run.smithi096.stdout:4578 read 0xd607673 thru 0xd6096af (0x203d bytes) 2024-09-22T01:15:25.480 INFO:teuthology.orchestra.run.smithi096.stdout:4579 write 0xa5b939 thru 0xa662d1 (0xa999 bytes) 2024-09-22T01:15:25.493 INFO:teuthology.orchestra.run.smithi096.stdout:4580 write 0x9698ecf thru 0x96a0d56 (0x7e88 bytes) 2024-09-22T01:15:25.503 INFO:teuthology.orchestra.run.smithi096.stdout:4581 writesame 0x5adaf51 thru 0x5ae2c04 data_size 0x2e(0x7cb4 bytes) 2024-09-22T01:15:25.515 INFO:teuthology.orchestra.run.smithi096.stdout:4584 writesame 0xd364210 thru 0xd3736a9 data_size 0x83(0xf49a bytes) 2024-09-22T01:15:25.528 INFO:teuthology.orchestra.run.smithi096.stdout:4585 read 0x70a56e8 thru 0x70aee3b (0x9754 bytes) 2024-09-22T01:15:25.535 INFO:teuthology.orchestra.run.smithi096.stdout:4586 trunc from 0xe5e0817 to 0x9c68f18 2024-09-22T01:15:25.557 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:25.553+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:25.561 INFO:teuthology.orchestra.run.smithi096.stdout:4587 read 0x53968b4 thru 0x539889e (0x1feb bytes) 2024-09-22T01:15:25.562 INFO:teuthology.orchestra.run.smithi096.stdout:4590 trunc from 0x9c68f18 to 0x8b9f781 2024-09-22T01:15:25.571 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.571 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.584 INFO:teuthology.orchestra.run.smithi096.stdout:4591 read 0x2254fe5 thru 0x2259adb (0x4af7 bytes) 2024-09-22T01:15:25.584 INFO:teuthology.orchestra.run.smithi096.stdout:4593 writesame 0x7670083 thru 0x767634d data_size 0x1(0x62cb bytes) 2024-09-22T01:15:25.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:25.611 INFO:teuthology.orchestra.run.smithi096.stdout:4594 read 0x1f250bc thru 0x1f26ba2 (0x1ae7 bytes) 2024-09-22T01:15:25.612 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.612 INFO:teuthology.orchestra.run.smithi096.stdout:4596 punch from 0x1a46f3f to 0x1a5572e, (0xe7ef bytes) 2024-09-22T01:15:25.612 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.612 INFO:teuthology.orchestra.run.smithi096.stdout:4600 punch from 0x500c74 to 0x5032e7, (0x2673 bytes) 2024-09-22T01:15:25.627 INFO:teuthology.orchestra.run.smithi096.stdout:4601 writesame 0xb7b4dfb thru 0xb7ba4b5 data_size 0x3(0x56bb bytes) 2024-09-22T01:15:25.627 INFO:teuthology.orchestra.run.smithi096.stdout:4604 punch from 0x5bbf958 to 0x5bcf1ce, (0xf876 bytes) 2024-09-22T01:15:25.627 INFO:teuthology.orchestra.run.smithi096.stdout:4605 write 0x614f9 thru 0x6768c (0x6194 bytes) 2024-09-22T01:15:25.636 INFO:teuthology.orchestra.run.smithi096.stdout:4606 trunc from 0xb7ba4b6 to 0x88b5277 2024-09-22T01:15:25.645 INFO:teuthology.orchestra.run.smithi096.stdout:4608 write 0x22918f7 thru 0x229f01f (0xd729 bytes) 2024-09-22T01:15:25.658 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:25.653+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:25.661 INFO:teuthology.orchestra.run.smithi096.stdout:4609 punch from 0x291e8e2 to 0x291f42a, (0xb48 bytes) 2024-09-22T01:15:25.661 INFO:teuthology.orchestra.run.smithi096.stdout:4610 punch from 0x25637b3 to 0x2565a7f, (0x22cc bytes) 2024-09-22T01:15:25.665 INFO:teuthology.orchestra.run.smithi096.stdout:4612 trunc from 0x88b5277 to 0x5bec5e3 2024-09-22T01:15:25.694 INFO:teuthology.orchestra.run.smithi096.stdout:4613 trunc from 0x5bec5e3 to 0x2eefe71 2024-09-22T01:15:25.739 INFO:teuthology.orchestra.run.smithi096.stdout:4615 trunc from 0x2eefe71 to 0xcb97d61 2024-09-22T01:15:25.745 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.745 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.750 INFO:teuthology.orchestra.run.smithi096.stdout:4616 punch from 0x1b53d33 to 0x1b55859, (0x1b26 bytes) 2024-09-22T01:15:25.750 INFO:teuthology.orchestra.run.smithi096.stdout:4617 writesame 0x3ca6736 thru 0x3cb3496 data_size 0x31(0xcd61 bytes) 2024-09-22T01:15:25.759 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:25.758+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:25.760 INFO:teuthology.orchestra.run.smithi096.stdout:4619 write 0x35eeb53 thru 0x35f1858 (0x2d06 bytes) 2024-09-22T01:15:25.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:25.768 INFO:teuthology.orchestra.run.smithi096.stdout:4622 read 0x86e3b68 thru 0x86e900a (0x54a3 bytes) 2024-09-22T01:15:25.768 INFO:teuthology.orchestra.run.smithi096.stdout:4624 write 0x7143dc thru 0x7199fe (0x5623 bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4625 punch from 0x376c87e to 0x376d4e6, (0xc68 bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4626 read 0xc23a8d6 thru 0xc243131 (0x885c bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4627 punch from 0x96cf94 to 0x96e165, (0x11d1 bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4628 read 0x34ed2c1 thru 0x34f9291 (0xbfd1 bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4629 read 0x6570ccc thru 0x657629e (0x55d3 bytes) 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.776 INFO:teuthology.orchestra.run.smithi096.stdout:4632 read 0x6f6a754 thru 0x6f7a631 (0xfede bytes) 2024-09-22T01:15:25.777 INFO:teuthology.orchestra.run.smithi096.stdout:4633 trunc from 0xcb97d61 to 0xdd0d073 2024-09-22T01:15:25.785 INFO:teuthology.orchestra.run.smithi096.stdout:4634 punch from 0x7a6d585 to 0x7a74778, (0x71f3 bytes) 2024-09-22T01:15:25.785 INFO:teuthology.orchestra.run.smithi096.stdout:4636 write 0x74f5014 thru 0x74fe9c6 (0x99b3 bytes) 2024-09-22T01:15:25.794 INFO:teuthology.orchestra.run.smithi096.stdout:4637 read 0x4f93e4e thru 0x4f93ec8 (0x7b bytes) 2024-09-22T01:15:25.795 INFO:teuthology.orchestra.run.smithi096.stdout:4638 write 0x2b222eb thru 0x2b31fe1 (0xfcf7 bytes) 2024-09-22T01:15:25.808 INFO:teuthology.orchestra.run.smithi096.stdout:4639 trunc from 0xdd0d073 to 0xc0155fd 2024-09-22T01:15:25.815 INFO:teuthology.orchestra.run.smithi096.stdout:4641 read 0x48330bd thru 0x483a871 (0x77b5 bytes) 2024-09-22T01:15:25.815 INFO:teuthology.orchestra.run.smithi096.stdout:4642 read 0x6b09faf thru 0x6b0d81b (0x386d bytes) 2024-09-22T01:15:25.815 INFO:teuthology.orchestra.run.smithi096.stdout:4645 write 0x643777a thru 0x643821f (0xaa6 bytes) 2024-09-22T01:15:25.823 INFO:teuthology.orchestra.run.smithi096.stdout:4646 write 0x97e27e9 thru 0x97ee622 (0xbe3a bytes) 2024-09-22T01:15:25.833 INFO:teuthology.orchestra.run.smithi096.stdout:4647 writesame 0x993a49d thru 0x993afa6 data_size 0x9(0xb0a bytes) 2024-09-22T01:15:25.844 INFO:teuthology.orchestra.run.smithi096.stdout:4651 trunc from 0xc0155fd to 0x842ce0b 2024-09-22T01:15:25.860 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:25.858+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:25.864 INFO:teuthology.orchestra.run.smithi096.stdout:4652 writesame 0x31a7472 thru 0x31a9ffd data_size 0x6(0x2b8c bytes) 2024-09-22T01:15:25.893 INFO:teuthology.orchestra.run.smithi096.stdout:4653 write 0xd116dca thru 0xd11a1ab (0x33e2 bytes) 2024-09-22T01:15:25.900 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:25.900 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:4654 punch from 0x7c3da16 to 0x7c44e0a, (0x73f4 bytes) 2024-09-22T01:15:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:4655 read 0x638e881 thru 0x639790e (0x908e bytes) 2024-09-22T01:15:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:4656 punch from 0x3abd8ad to 0x3ac1381, (0x3ad4 bytes) 2024-09-22T01:15:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:25.901 INFO:teuthology.orchestra.run.smithi096.stdout:4659 read 0x37e0348 thru 0x37ebfc7 (0xbc80 bytes) 2024-09-22T01:15:25.903 INFO:teuthology.orchestra.run.smithi096.stdout:4660 trunc from 0xd11a1ac to 0xe6d7b1e 2024-09-22T01:15:25.912 INFO:teuthology.orchestra.run.smithi096.stdout:4661 read 0xd3af0a4 thru 0xd3af6b8 (0x615 bytes) 2024-09-22T01:15:25.912 INFO:teuthology.orchestra.run.smithi096.stdout:4662 read 0x50e8fdc thru 0x50ef70c (0x6731 bytes) 2024-09-22T01:15:25.912 INFO:teuthology.orchestra.run.smithi096.stdout:4663 punch from 0xcc62082 to 0xcc6ce35, (0xadb3 bytes) 2024-09-22T01:15:25.912 INFO:teuthology.orchestra.run.smithi096.stdout:4664 punch from 0x3df050 to 0x3e1521, (0x24d1 bytes) 2024-09-22T01:15:25.913 INFO:teuthology.orchestra.run.smithi096.stdout:4665 write 0x1c973b0 thru 0x1ca1cab (0xa8fc bytes) 2024-09-22T01:15:25.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:25.946 INFO:teuthology.orchestra.run.smithi096.stdout:4667 trunc from 0xe6d7b1e to 0xdf801b 2024-09-22T01:15:25.962 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:25.958+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:25.976 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:25.976 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:25.976 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-22T01:15:25.977 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:15:25.977 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:15:25.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-22T01:15:25.982 INFO:teuthology.orchestra.run.smithi096.stdout:4669 read 0xbb416a thru 0xbb9259 (0x50f0 bytes) 2024-09-22T01:15:25.983 INFO:teuthology.orchestra.run.smithi096.stdout:4670 clone 38 order 19 su 65536 sc 5 2024-09-22T01:15:26.062 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:26.062+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:26.069 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:26.069 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:26.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:26.105 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:26.105 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:15:26.105 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:26.176 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:26.173+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:26.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:26.253 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:26.253 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:26.264 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:26.262+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:26.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:26.355 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:26.355 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:26.364 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:26.362+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:26.372 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:26.372 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:26.388 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:26.389 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:26.405 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:26.406 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:26.426 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:26.426 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:26.433 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:26.433 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:26.433 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:26.434 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:26.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:26.448 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:26.448 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:26.466 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:26.466+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:26.468 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:26.468 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:26.489 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:26.489 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:26.567 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:26.566+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:26.583 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:26.583 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:26.584 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:26.584 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:26.598 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:26.669 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:26.666+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:26.714 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.817149745999998, 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:26.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.816985389999999, 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:15:26.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:15:26.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812293614000001, 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:15:26.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812256059999999, 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:15:26.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:15:26.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812223897999999, 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:15:26.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812201560999998, 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812180201, 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.812159010999999, 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808253518000001, 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:26.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:15:26.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808231336999999, 2024-09-22T01:15:26.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:15:26.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:15:26.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808210059, 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:15:26.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:15:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808187711999999, 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:26.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:15:26.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808166035999999, 2024-09-22T01:15:26.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:15:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:15:26.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:15:26.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.808144614, 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:15:26.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:15:26.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.801505869, 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:15:26.777 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.801289122, 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.800906495, 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.800500955, 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:26.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.797284493999999, 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.760499099, 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:26.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:15:26.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:15:26.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:26.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:26.803 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:26.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:26.805 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:26.802+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:26.881 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:15:26.928 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:26.925+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:26.939 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:26.940 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:26.955 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:27.044 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:27.041+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:27.074 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-22T01:15:27.107 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:27.101+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:27.116 INFO:teuthology.orchestra.run.smithi096.stdout:4671 writesame 0x37e5ccf thru 0x37ef950 data_size 0x43(0x9c82 bytes) 2024-09-22T01:15:27.116 INFO:teuthology.orchestra.run.smithi096.stdout:4672 read 0x33b2bb4 thru 0x33bf065 (0xc4b2 bytes) 2024-09-22T01:15:27.119 INFO:teuthology.orchestra.run.smithi096.stdout:4674 trunc from 0x37ef951 to 0x1a42e63 2024-09-22T01:15:27.132 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:27.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:27.132 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:27.132 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:27.132 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:27.133 INFO:teuthology.orchestra.run.smithi096.stdout:4675 punch from 0x27d26f to 0x281d7c, (0x4b0d bytes) 2024-09-22T01:15:27.133 INFO:teuthology.orchestra.run.smithi096.stdout:4676 punch from 0x63a016 to 0x63b5f0, (0x15da bytes) 2024-09-22T01:15:27.146 INFO:teuthology.orchestra.run.smithi096.stdout:4677 trunc from 0x1a42e63 to 0xa8aa672 2024-09-22T01:15:27.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:27.159 INFO:teuthology.orchestra.run.smithi096.stdout:4679 punch from 0x67c91c3 to 0x67cffdb, (0x6e18 bytes) 2024-09-22T01:15:27.173 INFO:teuthology.orchestra.run.smithi096.stdout:4680 write 0xc55f156 thru 0xc569709 (0xa5b4 bytes) 2024-09-22T01:15:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:4681 write 0x7947aa5 thru 0x794fc5a (0x81b6 bytes) 2024-09-22T01:15:27.189 INFO:teuthology.orchestra.run.smithi096.stdout:4682 writesame 0x54020bf thru 0x541124c data_size 0x31(0xf18e bytes) 2024-09-22T01:15:27.199 INFO:teuthology.orchestra.run.smithi096.stdout:4685 writesame 0x55ac10a thru 0x55adbcc data_size 0x11(0x1ac3 bytes) 2024-09-22T01:15:27.207 INFO:teuthology.orchestra.run.smithi096.stdout:4686 read 0xc5ef2d thru 0xc62202 (0x32d6 bytes) 2024-09-22T01:15:27.207 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:27.205+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:27.208 INFO:teuthology.orchestra.run.smithi096.stdout:4687 writesame 0x9b3c20 thru 0x9c0d1a data_size 0x11(0xd0fb bytes) 2024-09-22T01:15:27.222 INFO:teuthology.orchestra.run.smithi096.stdout:4689 trunc from 0xc56970a to 0xd9a0060 2024-09-22T01:15:27.235 INFO:teuthology.orchestra.run.smithi096.stdout:4691 write 0x116d900 thru 0x1172cdf (0x53e0 bytes) 2024-09-22T01:15:27.244 INFO:teuthology.orchestra.run.smithi096.stdout:4693 write 0x27191f9 thru 0x271a68f (0x1497 bytes) 2024-09-22T01:15:27.250 INFO:teuthology.orchestra.run.smithi096.stdout:4694 read 0x2dd160d thru 0x2dd4792 (0x3186 bytes) 2024-09-22T01:15:27.250 INFO:teuthology.orchestra.run.smithi096.stdout:4695 write 0x12f32a5 thru 0x12ffa47 (0xc7a3 bytes) 2024-09-22T01:15:27.257 INFO:teuthology.orchestra.run.smithi096.stdout:4696 write 0xb68e53 thru 0xb7505b (0xc209 bytes) 2024-09-22T01:15:27.274 INFO:teuthology.orchestra.run.smithi096.stdout:4699 trunc from 0xd9a0060 to 0xeb397bf 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 92.262614083000003, 2024-09-22T01:15:27.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:27.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:27.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.715050286999997, 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.291 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:27.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:27.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.295 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 47.703739622000001, 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.465388537999999, 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135513064000001, 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135502042999999, 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135491218999999, 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:27.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135480340000001, 2024-09-22T01:15:27.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:27.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.321 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135466327, 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135435401999999, 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:27.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135410314000001, 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.334 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135386079, 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:27.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135364063000001, 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:27.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135334554, 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135282869000001, 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:27.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:27.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135250240000001, 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:27.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135192862, 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:27.356 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:27.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:27.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135177798000001, 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.364 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135164874000001, 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:27.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.135153275, 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:27.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:27.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:27.374 INFO:teuthology.orchestra.run.smithi096.stdout:4700 read 0xfa7142 thru 0xfb39ba (0xc879 bytes) 2024-09-22T01:15:27.374 INFO:teuthology.orchestra.run.smithi096.stdout:4701 punch from 0x1c82f0f to 0x1c91141, (0xe232 bytes) 2024-09-22T01:15:27.375 INFO:teuthology.orchestra.run.smithi096.stdout:4705 writesame 0xa17114b thru 0xa179ff0 data_size 0x26(0x8ea6 bytes) 2024-09-22T01:15:27.375 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:27.376 INFO:teuthology.orchestra.run.smithi096.stdout:4706 read 0xa084b6d thru 0xa09286d (0xdd01 bytes) 2024-09-22T01:15:27.376 INFO:teuthology.orchestra.run.smithi096.stdout:4707 trunc from 0xeb397bf to 0xe272ae9 2024-09-22T01:15:27.376 INFO:teuthology.orchestra.run.smithi096.stdout:4709 write 0xec00c99 thru 0xec033a7 (0x270f bytes) 2024-09-22T01:15:27.376 INFO:teuthology.orchestra.run.smithi096.stdout:4710 trunc from 0xec033a8 to 0xd05fbfd 2024-09-22T01:15:27.376 INFO:teuthology.orchestra.run.smithi096.stdout:4711 read 0x68c85e2 thru 0x68d5e89 (0xd8a8 bytes) 2024-09-22T01:15:27.377 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:27.369+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:27.382 INFO:teuthology.orchestra.run.smithi096.stdout:4712 write 0xd4960b1 thru 0xd4962c4 (0x214 bytes) 2024-09-22T01:15:27.389 INFO:teuthology.orchestra.run.smithi096.stdout:4713 read 0xaedceab thru 0xaeded4f (0x1ea5 bytes) 2024-09-22T01:15:27.389 INFO:teuthology.orchestra.run.smithi096.stdout:4715 writesame 0xb252165 thru 0xb2575fa data_size 0x1b(0x5496 bytes) 2024-09-22T01:15:27.397 INFO:teuthology.orchestra.run.smithi096.stdout:4716 write 0xbbf1fd0 thru 0xbbf399a (0x19cb bytes) 2024-09-22T01:15:27.402 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.402 INFO:teuthology.orchestra.run.smithi096.stdout:4720 write 0x21836ce thru 0x2184ed8 (0x180b bytes) 2024-09-22T01:15:27.408 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.408 INFO:teuthology.orchestra.run.smithi096.stdout:4722 read 0x390dfb2 thru 0x390f964 (0x19b3 bytes) 2024-09-22T01:15:27.408 INFO:teuthology.orchestra.run.smithi096.stdout:4723 write 0xae94716 thru 0xae95cae (0x1599 bytes) 2024-09-22T01:15:27.414 INFO:teuthology.orchestra.run.smithi096.stdout:4724 write 0x87977e6 thru 0x879e2d8 (0x6af3 bytes) 2024-09-22T01:15:27.451 INFO:teuthology.orchestra.run.smithi096.stdout:4725 trunc from 0xd4962c5 to 0x14274f0 2024-09-22T01:15:27.476 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:27.474+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:27.477 INFO:teuthology.orchestra.run.smithi096.stdout:4726 write 0x7e21a1 thru 0x7e6ebd (0x4d1d bytes) 2024-09-22T01:15:27.494 INFO:teuthology.orchestra.run.smithi096.stdout:4727 write 0x1bec73e thru 0x1bef0e2 (0x29a5 bytes) 2024-09-22T01:15:27.502 INFO:teuthology.orchestra.run.smithi096.stdout:4729 writesame 0x44b112 thru 0x4512a0 data_size 0x6f(0x618f bytes) 2024-09-22T01:15:27.517 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:27.517 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:27.518 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:27.518 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:27.532 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:27.543 INFO:teuthology.orchestra.run.smithi096.stdout:4731 writesame 0x8b3aaa7 thru 0x8b49bd5 data_size 0xb(0xf12f bytes) 2024-09-22T01:15:27.543 INFO:teuthology.orchestra.run.smithi096.stdout:4733 read 0x3a9d4d2 thru 0x3aac84b (0xf37a bytes) 2024-09-22T01:15:27.566 INFO:teuthology.orchestra.run.smithi096.stdout:4735 write 0xe7c4a28 thru 0xe7c9fa1 (0x557a bytes) 2024-09-22T01:15:27.573 INFO:teuthology.orchestra.run.smithi096.stdout:4736 read 0xdd147f7 thru 0xdd1b030 (0x683a bytes) 2024-09-22T01:15:27.573 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.586 INFO:teuthology.orchestra.run.smithi096.stdout:4739 trunc from 0xe7c9fa2 to 0x9b2cca5 2024-09-22T01:15:27.604 INFO:teuthology.orchestra.run.smithi096.stdout:4742 read 0x997cfbb thru 0x998c76a (0xf7b0 bytes) 2024-09-22T01:15:27.604 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.611 INFO:teuthology.orchestra.run.smithi096.stdout:4746 trunc from 0x9b2cca5 to 0x7156098 2024-09-22T01:15:27.617 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:27.618+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:27.636 INFO:teuthology.orchestra.run.smithi096.stdout:4747 trunc from 0x7156098 to 0x11e376e 2024-09-22T01:15:27.656 INFO:teuthology.orchestra.run.smithi096.stdout:4748 read 0x301dce thru 0x30a447 (0x867a bytes) 2024-09-22T01:15:27.677 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:27.678+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:27.701 INFO:teuthology.orchestra.run.smithi096.stdout:4750 write 0xe0f4a41 thru 0xe0f6634 (0x1bf4 bytes) 2024-09-22T01:15:27.702 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:27.702 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:27.702 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:27.702 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:27.702 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:27.711 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.711 INFO:teuthology.orchestra.run.smithi096.stdout:4753 read 0x1f3e44f thru 0x1f4558e (0x7140 bytes) 2024-09-22T01:15:27.711 INFO:teuthology.orchestra.run.smithi096.stdout:4754 write 0x20a2103 thru 0x20af887 (0xd785 bytes) 2024-09-22T01:15:27.718 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:27.719 INFO:teuthology.orchestra.run.smithi096.stdout:4755 write 0xbb10e3b thru 0xbb1b3c3 (0xa589 bytes) 2024-09-22T01:15:27.727 INFO:teuthology.orchestra.run.smithi096.stdout:4756 punch from 0x6b4b9e6 to 0x6b4e735, (0x2d4f bytes) 2024-09-22T01:15:27.727 INFO:teuthology.orchestra.run.smithi096.stdout:4757 read 0xb49c67b thru 0xb4a5e69 (0x97ef bytes) 2024-09-22T01:15:27.727 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:27.727 INFO:teuthology.orchestra.run.smithi096.stdout:4759 write 0x23b3415 thru 0x23be939 (0xb525 bytes) 2024-09-22T01:15:27.756 INFO:teuthology.orchestra.run.smithi096.stdout:4760 trunc from 0xe0f6635 to 0x46171c5 2024-09-22T01:15:27.776 INFO:teuthology.orchestra.run.smithi096.stdout:4761 read 0x84c7e8 thru 0x8599f9 (0xd212 bytes) 2024-09-22T01:15:27.788 INFO:teuthology.orchestra.run.smithi096.stdout:4764 write 0x5889b31 thru 0x5892b71 (0x9041 bytes) 2024-09-22T01:15:27.810 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:27.810+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:27.833 INFO:teuthology.orchestra.run.smithi096.stdout:4765 write 0xa2b7ca8 thru 0xa2c74bc (0xf815 bytes) 2024-09-22T01:15:27.843 INFO:teuthology.orchestra.run.smithi096.stdout:4767 read 0x4c9416 thru 0x4cdf04 (0x4aef bytes) 2024-09-22T01:15:27.843 INFO:teuthology.orchestra.run.smithi096.stdout:4768 punch from 0x32de3b7 to 0x32e57c0, (0x7409 bytes) 2024-09-22T01:15:27.843 INFO:teuthology.orchestra.run.smithi096.stdout:4773 read 0x2c3923a thru 0x2c41c7f (0x8a46 bytes) 2024-09-22T01:15:27.865 INFO:teuthology.orchestra.run.smithi096.stdout:4775 trunc from 0xa2c74bd to 0x208dd8c 2024-09-22T01:15:27.879 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:27.878+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:27.884 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.547010505999999, 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.907 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546980236000003, 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.908 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546950662999997, 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:27.915 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546896572000001, 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.916 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546840727999999, 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.920 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:27.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:27.924 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546782837999999, 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:27.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546721105000003, 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:27.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:27.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546625642000002, 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:27.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:27.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546563818999999, 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.937 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:27.941 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546472678999997, 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:27.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:27.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546410772999998, 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.946 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:27.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:27.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:27.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546350285999999, 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.950 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546292815999998, 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:27.958 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546235535999998, 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:27.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:27.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546176011999997, 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.546123127999998, 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:27.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 58.537010565999999, 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:27.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:27.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:27.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.09375138, 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:27.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.072679600000001, 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.886362041, 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:27.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:27.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:27.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:27.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:27.990 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:27.990 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:27.994 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:27.994 INFO:teuthology.orchestra.run.smithi096.stdout:4776 write 0x46264bf thru 0x4631345 (0xae87 bytes) 2024-09-22T01:15:27.994 INFO:teuthology.orchestra.run.smithi096.stdout:4777 write 0x862b527 thru 0x8632253 (0x6d2d bytes) 2024-09-22T01:15:27.994 INFO:teuthology.orchestra.run.smithi096.stdout:4778 trunc from 0x8632254 to 0x4ddd1d2 2024-09-22T01:15:27.995 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:27.985+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:27.997 INFO:teuthology.orchestra.run.smithi096.stdout:4779 write 0xd7cb5bd thru 0xd7d6182 (0xabc6 bytes) 2024-09-22T01:15:28.007 INFO:teuthology.orchestra.run.smithi096.stdout:4780 read 0x2de86d3 thru 0x2df4940 (0xc26e bytes) 2024-09-22T01:15:28.007 INFO:teuthology.orchestra.run.smithi096.stdout:4782 punch from 0x61b6cb1 to 0x61be637, (0x7986 bytes) 2024-09-22T01:15:28.007 INFO:teuthology.orchestra.run.smithi096.stdout:4783 write 0x3196b36 thru 0x319883d (0x1d08 bytes) 2024-09-22T01:15:28.015 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.015 INFO:teuthology.orchestra.run.smithi096.stdout:4785 write 0xa174e2c thru 0xa17daa2 (0x8c77 bytes) 2024-09-22T01:15:28.023 INFO:teuthology.orchestra.run.smithi096.stdout:4787 write 0xebcbd9 thru 0xecafe5 (0xe40d bytes) 2024-09-22T01:15:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:4788 read 0x2f3b8f0 thru 0x2f465ce (0xacdf bytes) 2024-09-22T01:15:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:4789 punch from 0x5bfcfd5 to 0x5bfddb2, (0xddd bytes) 2024-09-22T01:15:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:4790 punch from 0x3dcf4f6 to 0x3dd0d62, (0x186c bytes) 2024-09-22T01:15:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:4791 read 0x61bc33e thru 0x61c9f46 (0xdc09 bytes) 2024-09-22T01:15:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:4794 punch from 0xc34f50c to 0xc3537f5, (0x42e9 bytes) 2024-09-22T01:15:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:4797 punch from 0xd1fedb to 0xd2edda, (0xeeff bytes) 2024-09-22T01:15:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:4798 punch from 0x7d9de23 to 0x7d9f74d, (0x192a bytes) 2024-09-22T01:15:28.037 INFO:teuthology.orchestra.run.smithi096.stdout:4799 writesame 0x7ad3789 thru 0x7ad5a43 data_size 0x1(0x22bb bytes) 2024-09-22T01:15:28.069 INFO:teuthology.orchestra.run.smithi096.stdout:4800 write 0xe04f3d9 thru 0xe05099a (0x15c2 bytes) 2024-09-22T01:15:28.077 INFO:teuthology.orchestra.run.smithi096.stdout:4802 trunc from 0xe05099b to 0xe768465 2024-09-22T01:15:28.092 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:28.089+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:28.102 INFO:teuthology.orchestra.run.smithi096.stdout:4803 trunc from 0xe768465 to 0x871950a 2024-09-22T01:15:28.121 INFO:teuthology.orchestra.run.smithi096.stdout:4805 write 0x19c230c thru 0x19c5941 (0x3636 bytes) 2024-09-22T01:15:28.126 INFO:teuthology.orchestra.run.smithi096.stdout:4806 write 0x8e7096 thru 0x8ec278 (0x51e3 bytes) 2024-09-22T01:15:28.133 INFO:teuthology.orchestra.run.smithi096.stdout:4807 read 0x16cff4f thru 0x16d771c (0x77ce bytes) 2024-09-22T01:15:28.133 INFO:teuthology.orchestra.run.smithi096.stdout:4808 punch from 0x2918621 to 0x291d85b, (0x523a bytes) 2024-09-22T01:15:28.133 INFO:teuthology.orchestra.run.smithi096.stdout:4811 read 0x303c9ba thru 0x3049289 (0xc8d0 bytes) 2024-09-22T01:15:28.152 INFO:teuthology.orchestra.run.smithi096.stdout:4812 trunc from 0x871950a to 0x146394b 2024-09-22T01:15:28.172 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:28.173 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:28.173 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:28.173 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:28.177 INFO:teuthology.orchestra.run.smithi096.stdout:4814 punch from 0x134b112 to 0x1350d25, (0x5c13 bytes) 2024-09-22T01:15:28.177 INFO:teuthology.orchestra.run.smithi096.stdout:4816 read 0x5e8f6a thru 0x5e9715 (0x7ac bytes) 2024-09-22T01:15:28.188 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:28.193 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:28.189+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:28.203 INFO:teuthology.orchestra.run.smithi096.stdout:4817 trunc from 0x146394b to 0xb7d59e4 2024-09-22T01:15:28.214 INFO:teuthology.orchestra.run.smithi096.stdout:4818 write 0x3ff5b9b thru 0x3ff9603 (0x3a69 bytes) 2024-09-22T01:15:28.220 INFO:teuthology.orchestra.run.smithi096.stdout:4820 write 0x17e72cd thru 0x17f0d18 (0x9a4c bytes) 2024-09-22T01:15:28.229 INFO:teuthology.orchestra.run.smithi096.stdout:4821 write 0x852ef82 thru 0x8537421 (0x84a0 bytes) 2024-09-22T01:15:28.237 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.237 INFO:teuthology.orchestra.run.smithi096.stdout:4825 writesame 0x837bc8b thru 0x8385eaf data_size 0x1f(0xa225 bytes) 2024-09-22T01:15:28.246 INFO:teuthology.orchestra.run.smithi096.stdout:4827 read 0x4c7aff2 thru 0x4c7ea34 (0x3a43 bytes) 2024-09-22T01:15:28.246 INFO:teuthology.orchestra.run.smithi096.stdout:4828 read 0x21107f4 thru 0x211c627 (0xbe34 bytes) 2024-09-22T01:15:28.246 INFO:teuthology.orchestra.run.smithi096.stdout:4832 writesame 0x13316e2 thru 0x1334612 data_size 0x1(0x2f31 bytes) 2024-09-22T01:15:28.265 INFO:teuthology.orchestra.run.smithi096.stdout:4833 read 0x6be1c16 thru 0x6be6a29 (0x4e14 bytes) 2024-09-22T01:15:28.279 INFO:teuthology.orchestra.run.smithi096.stdout:4835 trunc from 0xb7d59e4 to 0x65fa9fb 2024-09-22T01:15:28.294 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:28.289+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:28.305 INFO:teuthology.orchestra.run.smithi096.stdout:4836 punch from 0x77826 to 0x77ca3, (0x47d bytes) 2024-09-22T01:15:28.317 INFO:teuthology.orchestra.run.smithi096.stdout:4837 writesame 0x69ea46f thru 0x69eb710 data_size 0x5(0x12a2 bytes) 2024-09-22T01:15:28.317 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.318 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.318 INFO:teuthology.orchestra.run.smithi096.stdout:4841 read 0x14fe852 thru 0x1502c3a (0x43e9 bytes) 2024-09-22T01:15:28.318 INFO:teuthology.orchestra.run.smithi096.stdout:4843 read 0x3bb5670 thru 0x3bbd4a2 (0x7e33 bytes) 2024-09-22T01:15:28.318 INFO:teuthology.orchestra.run.smithi096.stdout:4845 write 0x370aa55 thru 0x37164a0 (0xba4c bytes) 2024-09-22T01:15:28.324 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:28.324 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:28.324 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:28.324 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:28.324 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:28.339 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:28.349 INFO:teuthology.orchestra.run.smithi096.stdout:4846 writesame 0x9c44d0c thru 0x9c480ce data_size 0x7(0x33c3 bytes) 2024-09-22T01:15:28.349 INFO:teuthology.orchestra.run.smithi096.stdout:4847 write 0x79a2dfb thru 0x79b1c01 (0xee07 bytes) 2024-09-22T01:15:28.358 INFO:teuthology.orchestra.run.smithi096.stdout:4849 punch from 0x6497190 to 0x6498f44, (0x1db4 bytes) 2024-09-22T01:15:28.358 INFO:teuthology.orchestra.run.smithi096.stdout:4850 read 0x621645d thru 0x6217344 (0xee8 bytes) 2024-09-22T01:15:28.358 INFO:teuthology.orchestra.run.smithi096.stdout:4851 flatten 2024-09-22T01:15:28.395 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:28.394+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:28.415 INFO:teuthology.orchestra.run.smithi096.stdout:4854 read 0x210d58d thru 0x2110887 (0x32fb bytes) 2024-09-22T01:15:28.416 INFO:teuthology.orchestra.run.smithi096.stdout:4855 write 0x2c5b0eb thru 0x2c61d0a (0x6c20 bytes) 2024-09-22T01:15:28.434 INFO:teuthology.orchestra.run.smithi096.stdout:4856 writesame 0xaef7ad8 thru 0xaf00e91 data_size 0x21(0x93ba bytes) 2024-09-22T01:15:28.436 INFO:teuthology.orchestra.run.smithi096.stdout:4857 trunc from 0xaf00e92 to 0xa583e29 2024-09-22T01:15:28.444 INFO:teuthology.orchestra.run.smithi096.stdout:4858 read 0x8dec932 thru 0x8decccd (0x39c bytes) 2024-09-22T01:15:28.444 INFO:teuthology.orchestra.run.smithi096.stdout:4859 write 0x353e1fe thru 0x353e34a (0x14d bytes) 2024-09-22T01:15:28.451 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.455 INFO:teuthology.orchestra.run.smithi096.stdout:4861 trunc from 0xa583e29 to 0xc276be4 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:28.481 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.334659227, 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:28.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:28.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.334559287000001, 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.489 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.334490396, 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:28.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:28.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.334414333000002, 2024-09-22T01:15:28.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:28.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:28.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.334351545000001, 2024-09-22T01:15:28.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:28.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333720611, 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:28.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333691472000002, 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:28.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333662146999998, 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:28.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:28.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333631511, 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:28.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:28.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333595843000001, 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:28.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333566860000001, 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:28.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333532647999998, 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:28.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333463827999999, 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:28.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:28.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333363314, 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333295159999999, 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:28.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333211429999999, 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:28.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.333077551999999, 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.332950703000002, 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.332913747999999, 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:28.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.332862106, 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:28.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:28.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:28.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.563 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:28.563 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:28.563 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:28.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4862 trunc from 0xc276be4 to 0x13a755b 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4864 punch from 0x104cc85 to 0x1050ff4, (0x436f bytes) 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4865 writesame 0xd374ab1 thru 0xd375318 data_size 0x4(0x868 bytes) 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4866 read 0x3e0425 thru 0x3e1a90 (0x166c bytes) 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4867 punch from 0x9f4b7af to 0x9f55583, (0x9dd4 bytes) 2024-09-22T01:15:28.565 INFO:teuthology.orchestra.run.smithi096.stdout:4868 write 0x98531de thru 0x9854133 (0xf56 bytes) 2024-09-22T01:15:28.566 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:28.566 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:28.561+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:28.568 INFO:teuthology.orchestra.run.smithi096.stdout:4869 write 0xac86573 thru 0xac8bc30 (0x56be bytes) 2024-09-22T01:15:28.575 INFO:teuthology.orchestra.run.smithi096.stdout:4870 read 0xb7fbf38 thru 0xb8008a8 (0x4971 bytes) 2024-09-22T01:15:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:4871 trunc from 0xd375319 to 0x954beb4 2024-09-22T01:15:28.608 INFO:teuthology.orchestra.run.smithi096.stdout:4874 punch from 0x8c166f to 0x8cf5aa, (0xdf3b bytes) 2024-09-22T01:15:28.608 INFO:teuthology.orchestra.run.smithi096.stdout:4875 read 0x441747a thru 0x441a6bf (0x3246 bytes) 2024-09-22T01:15:28.608 INFO:teuthology.orchestra.run.smithi096.stdout:4876 read 0x6ed9a2e thru 0x6edfaf5 (0x60c8 bytes) 2024-09-22T01:15:28.608 INFO:teuthology.orchestra.run.smithi096.stdout:4877 writesame 0x35469df thru 0x354ddd9 data_size 0x3(0x73fb bytes) 2024-09-22T01:15:28.622 INFO:teuthology.orchestra.run.smithi096.stdout:4878 punch from 0x4d0517a to 0x4d0d1b5, (0x803b bytes) 2024-09-22T01:15:28.622 INFO:teuthology.orchestra.run.smithi096.stdout:4879 read 0x79b7515 thru 0x79b9de2 (0x28ce bytes) 2024-09-22T01:15:28.622 INFO:teuthology.orchestra.run.smithi096.stdout:4880 write 0x781a0ff thru 0x7828139 (0xe03b bytes) 2024-09-22T01:15:28.631 INFO:teuthology.orchestra.run.smithi096.stdout:4882 punch from 0x2c59aaf to 0x2c5b1c5, (0x1716 bytes) 2024-09-22T01:15:28.631 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.631 INFO:teuthology.orchestra.run.smithi096.stdout:4884 write 0x7ef078d thru 0x7ef831e (0x7b92 bytes) 2024-09-22T01:15:28.637 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:28.637 INFO:teuthology.orchestra.run.smithi096.stdout:4887 writesame 0x2ec5a25 thru 0x2ed18b6 data_size 0x5e(0xbe92 bytes) 2024-09-22T01:15:28.644 INFO:teuthology.orchestra.run.smithi096.stdout:4888 clone 39 order 22 su 16384 sc 2 2024-09-22T01:15:28.665 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:28.661+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:28.721 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:28.721 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:28.738 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:28.738 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:28.739 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:28.739 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:28.739 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:28.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:28.755 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:28.755 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:28.765 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:28.761+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:28.771 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:28.772 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:28.789 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:28.789 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:28.807 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:28.808 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:28.825 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:28.826 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:28.843 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:28.844 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:28.866 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:28.866+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:28.876 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:28.876 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:28.890 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:28.978 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:28.978+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:29.057 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.057 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.068 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:29.066+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:29.070 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:29.131 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-22T01:15:29.156 INFO:teuthology.orchestra.run.smithi096.stdout:4890 read 0x7b7640b thru 0x7b7908b (0x2c81 bytes) 2024-09-22T01:15:29.156 INFO:teuthology.orchestra.run.smithi096.stdout:4891 write 0x519d812 thru 0x51ab78f (0xdf7e bytes) 2024-09-22T01:15:29.168 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:29.165+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:29.170 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.171 INFO:teuthology.orchestra.run.smithi096.stdout:4893 trunc from 0x954beb4 to 0x9cf5db5 2024-09-22T01:15:29.180 INFO:teuthology.orchestra.run.smithi096.stdout:4894 punch from 0x4307638 to 0x4315216, (0xdbde bytes) 2024-09-22T01:15:29.180 INFO:teuthology.orchestra.run.smithi096.stdout:4895 write 0x28da9b5 thru 0x28e52f9 (0xa945 bytes) 2024-09-22T01:15:29.196 INFO:teuthology.orchestra.run.smithi096.stdout:4897 read 0x471ce80 thru 0x471e394 (0x1515 bytes) 2024-09-22T01:15:29.196 INFO:teuthology.orchestra.run.smithi096.stdout:4898 write 0x77f773 thru 0x78b46f (0xbcfd bytes) 2024-09-22T01:15:29.244 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.257 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.257 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.264 INFO:teuthology.orchestra.run.smithi096.stdout:4900 write 0xd99ff5d thru 0xd9a5728 (0x57cc bytes) 2024-09-22T01:15:29.269 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:29.270+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:29.271 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:29.275 INFO:teuthology.orchestra.run.smithi096.stdout:4901 read 0x198eb63 thru 0x19944f7 (0x5995 bytes) 2024-09-22T01:15:29.275 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.278 INFO:teuthology.orchestra.run.smithi096.stdout:4903 trunc from 0xd9a5729 to 0xeda6fbe 2024-09-22T01:15:29.291 INFO:teuthology.orchestra.run.smithi096.stdout:4905 punch from 0xd3bcf6b to 0xd3cb009, (0xe09e bytes) 2024-09-22T01:15:29.292 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.292 INFO:teuthology.orchestra.run.smithi096.stdout:4907 writesame 0x3294899 thru 0x32a170f data_size 0x37(0xce77 bytes) 2024-09-22T01:15:29.315 INFO:teuthology.orchestra.run.smithi096.stdout:4909 punch from 0x1e04044 to 0x1e052ec, (0x12a8 bytes) 2024-09-22T01:15:29.315 INFO:teuthology.orchestra.run.smithi096.stdout:4910 read 0x19622c4 thru 0x19720aa (0xfde7 bytes) 2024-09-22T01:15:29.315 INFO:teuthology.orchestra.run.smithi096.stdout:4911 read 0x31b7d77 thru 0x31b8082 (0x30c bytes) 2024-09-22T01:15:29.316 INFO:teuthology.orchestra.run.smithi096.stdout:4912 write 0x68a46ff thru 0x68a6f8d (0x288f bytes) 2024-09-22T01:15:29.325 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.326 INFO:teuthology.orchestra.run.smithi096.stdout:4914 read 0x8f6518f thru 0x8f67f20 (0x2d92 bytes) 2024-09-22T01:15:29.326 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:29.326 INFO:teuthology.orchestra.run.smithi096.stdout:4917 read 0x4df1d4f thru 0x4df4066 (0x2318 bytes) 2024-09-22T01:15:29.326 INFO:teuthology.orchestra.run.smithi096.stdout:4919 punch from 0xdc6215b to 0xdc70b41, (0xe9e6 bytes) 2024-09-22T01:15:29.326 INFO:teuthology.orchestra.run.smithi096.stdout:4921 write 0x83a54f thru 0x8477d7 (0xd289 bytes) 2024-09-22T01:15:29.370 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:29.370+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:29.377 INFO:teuthology.orchestra.run.smithi096.stdout:4922 read 0x77565c thru 0x77fe5a (0xa7ff bytes) 2024-09-22T01:15:29.378 INFO:teuthology.orchestra.run.smithi096.stdout:4927 read 0x28eba97 thru 0x28ef3d0 (0x393a bytes) 2024-09-22T01:15:29.383 INFO:teuthology.orchestra.run.smithi096.stdout:4928 trunc from 0xeda6fbe to 0xda7756c 2024-09-22T01:15:29.400 INFO:teuthology.orchestra.run.smithi096.stdout:4930 read 0x8d0a816 thru 0x8d141fd (0x99e8 bytes) 2024-09-22T01:15:29.405 INFO:teuthology.orchestra.run.smithi096.stdout:4931 trunc from 0xda7756c to 0xb86f702 2024-09-22T01:15:29.431 INFO:teuthology.orchestra.run.smithi096.stdout:4933 read 0x58031fd thru 0x5806d5a (0x3b5e bytes) 2024-09-22T01:15:29.432 INFO:teuthology.orchestra.run.smithi096.stdout:4938 writesame 0x8d1969c thru 0x8d26662 data_size 0x1(0xcfc7 bytes) 2024-09-22T01:15:29.447 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.447 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:29.502 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:29.502+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:29.526 INFO:teuthology.orchestra.run.smithi096.stdout:4940 read 0x942c504 thru 0x9433cf1 (0x77ee bytes) 2024-09-22T01:15:29.527 INFO:teuthology.orchestra.run.smithi096.stdout:4941 punch from 0x70a910e to 0x70a9892, (0x784 bytes) 2024-09-22T01:15:29.528 INFO:teuthology.orchestra.run.smithi096.stdout:4942 writesame 0x3bcd2d thru 0x3c11a6 data_size 0x5(0x447a bytes) 2024-09-22T01:15:29.540 INFO:teuthology.orchestra.run.smithi096.stdout:4944 trunc from 0xb86f702 to 0xac34147 2024-09-22T01:15:29.572 INFO:teuthology.orchestra.run.smithi096.stdout:4945 trunc from 0xac34147 to 0x6c3c4a4 2024-09-22T01:15:29.572 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:29.569+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:29.628 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.628 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.637 INFO:teuthology.orchestra.run.smithi096.stdout:4946 write 0xe2566e3 thru 0xe25c3cf (0x5ced bytes) 2024-09-22T01:15:29.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:29.646 INFO:teuthology.orchestra.run.smithi096.stdout:4948 write 0x8a233a6 thru 0x8a247b2 (0x140d bytes) 2024-09-22T01:15:29.669 INFO:teuthology.orchestra.run.smithi096.stdout:4949 trunc from 0xe25c3d0 to 0x8189fbd 2024-09-22T01:15:29.673 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:29.670+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:29.700 INFO:teuthology.orchestra.run.smithi096.stdout:4952 write 0xa644622 thru 0xa64becc (0x78ab bytes) 2024-09-22T01:15:29.706 INFO:teuthology.orchestra.run.smithi096.stdout:4953 write 0x2aae612 thru 0x2ab62b7 (0x7ca6 bytes) 2024-09-22T01:15:29.708 INFO:teuthology.orchestra.run.smithi096.stdout:4954 clone 40 order 25 su 8388608 sc 13 2024-09-22T01:15:29.775 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:29.769+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:29.775 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.775 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:29.880 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:29.877+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:29.963 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:29.963 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:29.976 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:29.974+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:29.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:29.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:15:30.023 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone39 from 0xa64becd (overlap 0x6c3c4a4) to 0x2b4b50e 2024-09-22T01:15:30.077 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:30.073+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:30.157 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.157 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.172 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-22T01:15:30.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:30.178 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:30.178+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:30.280 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:30.278+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:30.344 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.344 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:30.381 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:30.378+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:30.474 INFO:teuthology.orchestra.run.smithi096.stdout:4956 trunc from 0xa64becd to 0x3017486 2024-09-22T01:15:30.482 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:30.482+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:30.485 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.485 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.511 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:30.567 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:30.577 INFO:teuthology.orchestra.run.smithi096.stdout:4958 write 0x620ba88 thru 0x6213bd5 (0x814e bytes) 2024-09-22T01:15:30.582 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:30.577+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:30.598 INFO:teuthology.orchestra.run.smithi096.stdout:4959 write 0xc7c81a4 thru 0xc7d55c8 (0xd425 bytes) 2024-09-22T01:15:30.604 INFO:teuthology.orchestra.run.smithi096.stdout:4961 writesame 0x45e7981 thru 0x45e96a2 data_size 0x21(0x1d22 bytes) 2024-09-22T01:15:30.610 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:30.610 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:30.610 INFO:teuthology.orchestra.run.smithi096.stdout:4965 write 0x19bfbfb thru 0x19c8292 (0x8698 bytes) 2024-09-22T01:15:30.613 INFO:teuthology.orchestra.run.smithi096.stdout:4966 write 0xac7feeb thru 0xac8e1fd (0xe313 bytes) 2024-09-22T01:15:30.617 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:30.617 INFO:teuthology.orchestra.run.smithi096.stdout:4968 punch from 0xb9eb909 to 0xb9f53bc, (0x9ab3 bytes) 2024-09-22T01:15:30.623 INFO:teuthology.orchestra.run.smithi096.stdout:4970 trunc from 0xc7d55c9 to 0x87990f6 2024-09-22T01:15:30.636 INFO:teuthology.orchestra.run.smithi096.stdout:4971 read 0x48a5d04 thru 0x48ae2fc (0x85f9 bytes) 2024-09-22T01:15:30.640 INFO:teuthology.orchestra.run.smithi096.stdout:4972 trunc from 0x87990f6 to 0x6619362 2024-09-22T01:15:30.648 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.648 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.663 INFO:teuthology.orchestra.run.smithi096.stdout:4973 write 0x74e86f5 thru 0x74e96f4 (0x1000 bytes) 2024-09-22T01:15:30.665 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:30.666 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:30.676 INFO:teuthology.orchestra.run.smithi096.stdout:4975 write 0x9c5e535 thru 0x9c69d96 (0xb862 bytes) 2024-09-22T01:15:30.679 INFO:teuthology.orchestra.run.smithi096.stdout:4976 read 0x6a7908c thru 0x6a7e5a6 (0x551b bytes) 2024-09-22T01:15:30.680 INFO:teuthology.orchestra.run.smithi096.stdout:4977 read 0x4aa24cd thru 0x4ab14e5 (0xf019 bytes) 2024-09-22T01:15:30.683 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:30.677+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:30.693 INFO:teuthology.orchestra.run.smithi096.stdout:4978 writesame 0xe237de4 thru 0xe243ecd data_size 0x3(0xc0ea bytes) 2024-09-22T01:15:30.694 INFO:teuthology.orchestra.run.smithi096.stdout:4979 write 0x2172ddc thru 0x218293d (0xfb62 bytes) 2024-09-22T01:15:30.698 INFO:teuthology.orchestra.run.smithi096.stdout:4981 write 0x38b9608 thru 0x38bda2c (0x4425 bytes) 2024-09-22T01:15:30.707 INFO:teuthology.orchestra.run.smithi096.stdout:4982 write 0x8151060 thru 0x815180b (0x7ac bytes) 2024-09-22T01:15:30.709 INFO:teuthology.orchestra.run.smithi096.stdout:4985 clone 41 order 22 su 1048576 sc 7 2024-09-22T01:15:30.784 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:30.782+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:30.800 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.800 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.815 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:30.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:30.885 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:30.886+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:30.913 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:15:30.995 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:30.995 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:30.995 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:30.989+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:31.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:31.026 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone40 from 0xe243ece (overlap 0x3017486) to 0x267906b 2024-09-22T01:15:31.064 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:31.081 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:31.082 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:31.087 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:31.081+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:31.100 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:31.101 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:31.119 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:31.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:31.119 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-22T01:15:31.119 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-09-22T01:15:31.120 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0 2024-09-22T01:15:31.121 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:31.121 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:31.143 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:31.143 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:31.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.266380931999997, 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:15:31.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:15:31.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.266216575999998, 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:15:31.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261524799999997, 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:15:31.173 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:31.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:15:31.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:31.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261487246000002, 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:15:31.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261455083999998, 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:15:31.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261432747000001, 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:31.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261411387000003, 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:31.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.261390196999997, 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:15:31.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:15:31.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:15:31.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:15:31.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257484703999999, 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257462523000001, 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:31.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:15:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:15:31.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257441245000003, 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:15:31.209 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:15:31.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257418897999997, 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.214 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:15:31.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257397222000002, 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:31.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:15:31.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:15:31.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.257375799999998, 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:15:31.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:15:31.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.250737055000002, 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:15:31.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.250520307999999, 2024-09-22T01:15:31.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:31.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:15:31.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.250137680999998, 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:15:31.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.249732141000003, 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:15:31.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:31.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.246515680000002, 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:15:31.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:15:31.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.209730284999999, 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:15:31.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:15:31.253 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.254 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:31.255 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:31.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:31.256 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-22T01:15:31.256 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:31.257 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:31.257 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:31.258 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:31.258 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:31.254+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:31.272 INFO:teuthology.orchestra.run.smithi096.stdout:4986 write 0xd1a5330 thru 0xd1b4fa5 (0xfc76 bytes) 2024-09-22T01:15:31.283 INFO:teuthology.orchestra.run.smithi096.stdout:4988 write 0x5a9c782 thru 0x5aa04f9 (0x3d78 bytes) 2024-09-22T01:15:31.288 INFO:teuthology.orchestra.run.smithi096.stdout:4989 writesame 0x905ce0a thru 0x9063631 data_size 0x65(0x6828 bytes) 2024-09-22T01:15:31.297 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.297 INFO:teuthology.orchestra.run.smithi096.stdout:4991 write 0x1403bbb thru 0x1413938 (0xfd7e bytes) 2024-09-22T01:15:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:4992 write 0xb2caad thru 0xb3bf95 (0xf4e9 bytes) 2024-09-22T01:15:31.325 INFO:teuthology.orchestra.run.smithi096.stdout:4993 punch from 0x463671a to 0x4640951, (0xa237 bytes) 2024-09-22T01:15:31.326 INFO:teuthology.orchestra.run.smithi096.stdout:4994 read 0xdb5ac1 thru 0xdbd40e (0x794e bytes) 2024-09-22T01:15:31.326 INFO:teuthology.orchestra.run.smithi096.stdout:4995 writesame 0x60492de thru 0x604bd4e data_size 0x29(0x2a71 bytes) 2024-09-22T01:15:31.331 INFO:teuthology.orchestra.run.smithi096.stdout:4996 write 0x4ae65dc thru 0x4ae9bae (0x35d3 bytes) 2024-09-22T01:15:31.337 INFO:teuthology.orchestra.run.smithi096.stdout:4997 read 0x71e752a thru 0x71f6c1c (0xf6f3 bytes) 2024-09-22T01:15:31.338 INFO:teuthology.orchestra.run.smithi096.stdout:4998 write 0x4685f87 thru 0x468ab2d (0x4ba7 bytes) 2024-09-22T01:15:31.347 INFO:teuthology.orchestra.run.smithi096.stdout:4999 read 0xaa1cbbc thru 0xaa28148 (0xb58d bytes) 2024-09-22T01:15:31.348 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.348 INFO:teuthology.orchestra.run.smithi096.stdout:5002 read 0x2c5ed61 thru 0x2c6a795 (0xba35 bytes) 2024-09-22T01:15:31.349 INFO:teuthology.orchestra.run.smithi096.stdout:5003 read 0x16c02d6 thru 0x16c890d (0x8638 bytes) 2024-09-22T01:15:31.350 INFO:teuthology.orchestra.run.smithi096.stdout:5004 punch from 0xb955946 to 0xb9627c1, (0xce7b bytes) 2024-09-22T01:15:31.350 INFO:teuthology.orchestra.run.smithi096.stdout:5005 write 0x1d8f77b thru 0x1d93ef8 (0x477e bytes) 2024-09-22T01:15:31.356 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:31.354+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:31.357 INFO:teuthology.orchestra.run.smithi096.stdout:5006 read 0xdfd048 thru 0xdfd9b7 (0x970 bytes) 2024-09-22T01:15:31.357 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.357 INFO:teuthology.orchestra.run.smithi096.stdout:5008 punch from 0xa4b35c0 to 0xa4b470e, (0x114e bytes) 2024-09-22T01:15:31.358 INFO:teuthology.orchestra.run.smithi096.stdout:5009 write 0xce4468c thru 0xce467df (0x2154 bytes) 2024-09-22T01:15:31.362 INFO:teuthology.orchestra.run.smithi096.stdout:5010 write 0x10a36ec thru 0x10ad24e (0x9b63 bytes) 2024-09-22T01:15:31.381 INFO:teuthology.orchestra.run.smithi096.stdout:5011 write 0x5ab0040 thru 0x5abca1c (0xc9dd bytes) 2024-09-22T01:15:31.383 INFO:teuthology.orchestra.run.smithi096.stdout:5012 writesame 0x3c2061d thru 0x3c25e33 data_size 0x1(0x5817 bytes) 2024-09-22T01:15:31.399 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:31.399 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:31.399 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:31.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:31.405 INFO:teuthology.orchestra.run.smithi096.stdout:5013 write 0xb8b8a45 thru 0xb8c6d71 (0xe32d bytes) 2024-09-22T01:15:31.414 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:31.430 INFO:teuthology.orchestra.run.smithi096.stdout:5014 trunc from 0xe243ece to 0x226a2c6 2024-09-22T01:15:31.467 INFO:teuthology.orchestra.run.smithi096.stdout:5015 punch from 0x86c400 to 0x872ab8, (0x66b8 bytes) 2024-09-22T01:15:31.467 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:31.461+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:31.475 INFO:teuthology.orchestra.run.smithi096.stdout:5017 writesame 0x3c918e2 thru 0x3c9701d data_size 0x6(0x573c bytes) 2024-09-22T01:15:31.475 INFO:teuthology.orchestra.run.smithi096.stdout:5018 read 0x5e4a63 thru 0x5e9c6b (0x5209 bytes) 2024-09-22T01:15:31.476 INFO:teuthology.orchestra.run.smithi096.stdout:5019 read 0x30a2ec5 thru 0x30a64c7 (0x3603 bytes) 2024-09-22T01:15:31.486 INFO:teuthology.orchestra.run.smithi096.stdout:5020 write 0x58334b9 thru 0x5842d1a (0xf862 bytes) 2024-09-22T01:15:31.491 INFO:teuthology.orchestra.run.smithi096.stdout:5021 read 0x49c7105 thru 0x49cd3f9 (0x62f5 bytes) 2024-09-22T01:15:31.501 INFO:teuthology.orchestra.run.smithi096.stdout:5022 trunc from 0x5842d1b to 0xdfb2dd4 2024-09-22T01:15:31.509 INFO:teuthology.orchestra.run.smithi096.stdout:5023 writesame 0x558446b thru 0x5584879 data_size 0x1(0x40f bytes) 2024-09-22T01:15:31.514 INFO:teuthology.orchestra.run.smithi096.stdout:5024 read 0x587fc7c thru 0x588f8d9 (0xfc5e bytes) 2024-09-22T01:15:31.515 INFO:teuthology.orchestra.run.smithi096.stdout:5025 read 0x5836ba5 thru 0x58407a5 (0x9c01 bytes) 2024-09-22T01:15:31.515 INFO:teuthology.orchestra.run.smithi096.stdout:5026 write 0x4dda676 thru 0x4de8e0d (0xe798 bytes) 2024-09-22T01:15:31.521 INFO:teuthology.orchestra.run.smithi096.stdout:5028 writesame 0x5d29cc2 thru 0x5d35392 data_size 0x1(0xb6d1 bytes) 2024-09-22T01:15:31.553 INFO:teuthology.orchestra.run.smithi096.stdout:5029 punch from 0xbd3c23 to 0xbe3b09, (0xfee6 bytes) 2024-09-22T01:15:31.557 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:31.553+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:31.561 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:31.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:31.561 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:31.561 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:31.561 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:31.566 INFO:teuthology.orchestra.run.smithi096.stdout:5031 write 0xe061118 thru 0xe0646c6 (0x35af bytes) 2024-09-22T01:15:31.572 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.572 INFO:teuthology.orchestra.run.smithi096.stdout:5034 punch from 0x76496b to 0x769aaa, (0x513f bytes) 2024-09-22T01:15:31.572 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.572 INFO:teuthology.orchestra.run.smithi096.stdout:5036 writesame 0x872dd4d thru 0x87305c7 data_size 0x1(0x287b bytes) 2024-09-22T01:15:31.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:31.582 INFO:teuthology.orchestra.run.smithi096.stdout:5037 write 0x1d9ad9b thru 0x1d9e95a (0x3bc0 bytes) 2024-09-22T01:15:31.586 INFO:teuthology.orchestra.run.smithi096.stdout:5038 write 0x73815a0 thru 0x738371a (0x217b bytes) 2024-09-22T01:15:31.591 INFO:teuthology.orchestra.run.smithi096.stdout:5039 write 0x73b00cf thru 0x73b36be (0x35f0 bytes) 2024-09-22T01:15:31.598 INFO:teuthology.orchestra.run.smithi096.stdout:5040 trunc from 0xe0646c7 to 0xb268515 2024-09-22T01:15:31.613 INFO:teuthology.orchestra.run.smithi096.stdout:5042 read 0x2ce4da3 thru 0x2cf2041 (0xd29f bytes) 2024-09-22T01:15:31.613 INFO:teuthology.orchestra.run.smithi096.stdout:5043 write 0x82292fd thru 0x822e75b (0x545f bytes) 2024-09-22T01:15:31.618 INFO:teuthology.orchestra.run.smithi096.stdout:5045 punch from 0x50dcc6c to 0x50de732, (0x1ac6 bytes) 2024-09-22T01:15:31.618 INFO:teuthology.orchestra.run.smithi096.stdout:5046 punch from 0x96f33a1 to 0x96fca1c, (0x967b bytes) 2024-09-22T01:15:31.618 INFO:teuthology.orchestra.run.smithi096.stdout:5047 read 0x6b0a137 thru 0x6b17be0 (0xdaaa bytes) 2024-09-22T01:15:31.619 INFO:teuthology.orchestra.run.smithi096.stdout:5048 read 0x68b3486 thru 0x68c175c (0xe2d7 bytes) 2024-09-22T01:15:31.628 INFO:teuthology.orchestra.run.smithi096.stdout:5052 write 0xbcc2166 thru 0xbcce965 (0xc800 bytes) 2024-09-22T01:15:31.632 INFO:teuthology.orchestra.run.smithi096.stdout:5053 write 0x12c47df thru 0x12cdb12 (0x9334 bytes) 2024-09-22T01:15:31.637 INFO:teuthology.orchestra.run.smithi096.stdout:5054 punch from 0x4586904 to 0x458ea1a, (0x8116 bytes) 2024-09-22T01:15:31.637 INFO:teuthology.orchestra.run.smithi096.stdout:5055 read 0xa157c28 thru 0xa163861 (0xbc3a bytes) 2024-09-22T01:15:31.637 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.637 INFO:teuthology.orchestra.run.smithi096.stdout:5058 writesame 0x9c5ceca thru 0x9c5fda7 data_size 0x7(0x2ede bytes) 2024-09-22T01:15:31.644 INFO:teuthology.orchestra.run.smithi096.stdout:5059 write 0xf51f28 thru 0xf5f272 (0xd34b bytes) 2024-09-22T01:15:31.658 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:31.653+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:31.666 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.666 INFO:teuthology.orchestra.run.smithi096.stdout:5061 trunc from 0xbcce966 to 0xb747a0a 2024-09-22T01:15:31.677 INFO:teuthology.orchestra.run.smithi096.stdout:5062 writesame 0xb5fa7bf thru 0xb6075f0 data_size 0x1(0xce32 bytes) 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.686870026999998, 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:31.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:31.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:31.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:31.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:31.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 57.139306230999999, 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:31.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:31.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:31.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 52.127995566000003, 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:31.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:31.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:31.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.889644482000001, 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:31.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559769008, 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:31.730 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559757987000001, 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:31.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559747163000001, 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:31.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559736284, 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559722270999998, 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:31.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559691346000001, 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:31.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:31.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559666258, 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:31.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:31.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559642022999999, 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.764 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559620006999999, 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:31.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:31.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:31.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559590497999999, 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:31.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:31.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559538813, 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:31.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559506184, 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:31.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559448805999999, 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559433742, 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:31.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559420818, 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:31.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:31.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:31.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.559409218999999, 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:31.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:31.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:31.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:31.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:31.799 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:31.800 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:5064 read 0x4712212 thru 0x4712e80 (0xc6f bytes) 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:5068 punch from 0x169fcac to 0x16aa2b4, (0xa608 bytes) 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:5069 trunc from 0xb747a0a to 0xca2838f 2024-09-22T01:15:31.801 INFO:teuthology.orchestra.run.smithi096.stdout:5071 read 0x63769d9 thru 0x6384308 (0xd930 bytes) 2024-09-22T01:15:31.802 INFO:teuthology.orchestra.run.smithi096.stdout:5073 trunc from 0xca2838f to 0x7ada633 2024-09-22T01:15:31.802 INFO:teuthology.orchestra.run.smithi096.stdout:5074 trunc from 0x7ada633 to 0xd9082a2 2024-09-22T01:15:31.802 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:31.797+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:31.816 INFO:teuthology.orchestra.run.smithi096.stdout:5075 trunc from 0xd9082a2 to 0x71f69d8 2024-09-22T01:15:31.844 INFO:teuthology.orchestra.run.smithi096.stdout:5076 trunc from 0x71f69d8 to 0x469cb84 2024-09-22T01:15:31.887 INFO:teuthology.orchestra.run.smithi096.stdout:5077 write 0x93eb092 thru 0x93f2a84 (0x79f3 bytes) 2024-09-22T01:15:31.896 INFO:teuthology.orchestra.run.smithi096.stdout:5079 writesame 0x4201da5 thru 0x4203f6a data_size 0x21(0x21c6 bytes) 2024-09-22T01:15:31.900 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:31.898+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:31.915 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.4 primary-affinity to 0 (802) 2024-09-22T01:15:31.940 INFO:teuthology.orchestra.run.smithi096.stdout:5080 write 0xe768d92 thru 0xe7715b1 (0x8820 bytes) 2024-09-22T01:15:31.943 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:31.943 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:31.943 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:31.957 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:32.002 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:31.997+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:32.103 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:32.097+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:32.123 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:32.123 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:32.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:32.203 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:32.202+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.264 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907792743000002, 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.266 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907762472999998, 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.269 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:32.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:32.273 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907732899999999, 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:32.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:32.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907678808999997, 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.278 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:32.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:32.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:32.281 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907622965000002, 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907565075000001, 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:32.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.290 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907503341999998, 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:32.294 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907407878999997, 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.299 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907346056000002, 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:32.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.303 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907254915999999, 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.90719301, 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.308 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.312 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907132523000001, 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:32.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907075053, 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:32.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.322 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.907017773, 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:32.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.326 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.906958248999999, 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:32.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.906905365, 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:32.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:32.335 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.897792803000002, 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:32.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.454533616999999, 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:32.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 27.433461836999999, 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:32.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:32.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:32.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 19.247144278, 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:32.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.354 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.355 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:32.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:32.356 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:32.317+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:32.421 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:32.417+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:32.487 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:32.487 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:32.487 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:32.487 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:32.501 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:32.522 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:32.517+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:32.621 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:32.621 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:32.621 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:32.621 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:32.621 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:32.622 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:32.622+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:32.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:32.723 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:32.722+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:32.803 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.657465596000002, 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:32.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.657365656, 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:32.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:32.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.657296765000002, 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:32.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:32.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.657220702, 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.657157913999999, 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656526979999999, 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656497841, 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656468516, 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.833 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:32.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:32.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656437879999999, 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.837 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656402212, 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:32.841 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:32.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:32.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656373229, 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:32.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656339017000001, 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:32.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:32.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:32.854 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656270197000001, 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656169683000002, 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656101529000001, 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:32.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:32.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.656017799000001, 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.867 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:32.871 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.655883921000001, 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.875 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.655757072, 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:32.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:32.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:32.879 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.655720117000001, 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:32.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:32.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.655668474999999, 2024-09-22T01:15:32.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:32.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:32.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:32.887 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:32.889 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:32.889 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:32.886+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:32.948 INFO:teuthology.orchestra.run.smithi096.stdout:5082 read 0xe126776 thru 0xe12dc07 (0x7492 bytes) 2024-09-22T01:15:32.948 INFO:teuthology.orchestra.run.smithi096.stdout:5084 read 0x26dcfb6 thru 0x26dda6e (0xab9 bytes) 2024-09-22T01:15:32.949 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:5085 punch from 0x2f4b23f to 0x2f50524, (0x52e5 bytes) 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:5089 punch from 0xd1e14b1 to 0xd1e7d74, (0x68c3 bytes) 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:5090 read 0x4713323 thru 0x4720059 (0xcd37 bytes) 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:5091 punch from 0x339870 to 0x339b59, (0x2e9 bytes) 2024-09-22T01:15:32.950 INFO:teuthology.orchestra.run.smithi096.stdout:5092 read 0xdeac53 thru 0xdf840f (0xd7bd bytes) 2024-09-22T01:15:32.955 INFO:teuthology.orchestra.run.smithi096.stdout:5093 read 0xc30b3ff thru 0xc30f110 (0x3d12 bytes) 2024-09-22T01:15:32.955 INFO:teuthology.orchestra.run.smithi096.stdout:5094 write 0x72904e6 thru 0x729b016 (0xab31 bytes) 2024-09-22T01:15:32.972 INFO:teuthology.orchestra.run.smithi096.stdout:5095 write 0xe9691d2 thru 0xe96aa03 (0x1832 bytes) 2024-09-22T01:15:32.977 INFO:teuthology.orchestra.run.smithi096.stdout:5096 writesame 0xba091bc thru 0xba0bce9 data_size 0x1(0x2b2e bytes) 2024-09-22T01:15:32.989 INFO:teuthology.orchestra.run.smithi096.stdout:5097 read 0x590d119 thru 0x59104da (0x33c2 bytes) 2024-09-22T01:15:32.989 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:32.990+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:33.015 INFO:teuthology.orchestra.run.smithi096.stdout:5098 trunc from 0xe96aa04 to 0xa90263 2024-09-22T01:15:33.062 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:33.063 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:33.063 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:33.063 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:33.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:33.090 INFO:teuthology.orchestra.run.smithi096.stdout:5099 write 0x864ea00 thru 0x864eddb (0x3dc bytes) 2024-09-22T01:15:33.095 INFO:teuthology.orchestra.run.smithi096.stdout:5100 read 0x79835d4 thru 0x798b657 (0x8084 bytes) 2024-09-22T01:15:33.095 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.095 INFO:teuthology.orchestra.run.smithi096.stdout:5104 write 0x227eb51 thru 0x22806f2 (0x1ba2 bytes) 2024-09-22T01:15:33.110 INFO:teuthology.orchestra.run.smithi096.stdout:5105 writesame 0xab4b1ba thru 0xab4e62e data_size 0x1(0x3475 bytes) 2024-09-22T01:15:33.110 INFO:teuthology.orchestra.run.smithi096.stdout:5106 read 0x6d42730 thru 0x6d49aab (0x737c bytes) 2024-09-22T01:15:33.110 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.118 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:33.118+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:33.123 INFO:teuthology.orchestra.run.smithi096.stdout:5109 trunc from 0xab4e62f to 0x12547cc 2024-09-22T01:15:33.156 INFO:teuthology.orchestra.run.smithi096.stdout:5110 trunc from 0x12547cc to 0xb76abe7 2024-09-22T01:15:33.167 INFO:teuthology.orchestra.run.smithi096.stdout:5111 punch from 0x4a3ba24 to 0x4a4a556, (0xeb32 bytes) 2024-09-22T01:15:33.167 INFO:teuthology.orchestra.run.smithi096.stdout:5112 read 0x137f572 thru 0x138e140 (0xebcf bytes) 2024-09-22T01:15:33.167 INFO:teuthology.orchestra.run.smithi096.stdout:5113 read 0x94e49dd thru 0x94ef108 (0xa72c bytes) 2024-09-22T01:15:33.176 INFO:teuthology.orchestra.run.smithi096.stdout:5115 write 0xc5677db thru 0xc56d2a1 (0x5ac7 bytes) 2024-09-22T01:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:5116 punch from 0x2c636e to 0x2d14ec, (0xb17e bytes) 2024-09-22T01:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:5117 read 0xdcb57a thru 0xdd2ccf (0x7756 bytes) 2024-09-22T01:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:5118 punch from 0x8f86f45 to 0x8f8969e, (0x2759 bytes) 2024-09-22T01:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:5120 read 0xc02782a thru 0xc029c51 (0x2428 bytes) 2024-09-22T01:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:5121 write 0x9fd7cbe thru 0x9fdbf08 (0x424b bytes) 2024-09-22T01:15:33.190 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:33.190+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:33.205 INFO:teuthology.orchestra.run.smithi096.stdout:5122 trunc from 0xc56d2a2 to 0x303f290 2024-09-22T01:15:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:5123 trunc from 0x303f290 to 0x68370c6 2024-09-22T01:15:33.244 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:33.317 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:33.317 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:33.317 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:33.317 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:33.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:5124 writesame 0x9c118fe thru 0x9c1b3a4 data_size 0x53(0x9aa7 bytes) 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:5126 trunc from 0x9c1b3a5 to 0x47a9a55 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:5128 trunc from 0x47a9a55 to 0x5f344db 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:5131 write 0x8318067 thru 0x8326519 (0xe4b3 bytes) 2024-09-22T01:15:33.320 INFO:teuthology.orchestra.run.smithi096.stdout:5132 trunc from 0x832651a to 0x680aa54 2024-09-22T01:15:33.321 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:33.321 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:33.313+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:33.343 INFO:teuthology.orchestra.run.smithi096.stdout:5134 write 0xba672f5 thru 0xba68bd8 (0x18e4 bytes) 2024-09-22T01:15:33.349 INFO:teuthology.orchestra.run.smithi096.stdout:5135 trunc from 0xba68bd9 to 0xaf49e25 2024-09-22T01:15:33.360 INFO:teuthology.orchestra.run.smithi096.stdout:5136 trunc from 0xaf49e25 to 0xb1f4d8f 2024-09-22T01:15:33.368 INFO:teuthology.orchestra.run.smithi096.stdout:5137 writesame 0x65ebf16 thru 0x65f325f data_size 0x3(0x734a bytes) 2024-09-22T01:15:33.391 INFO:teuthology.orchestra.run.smithi096.stdout:5138 write 0xeae6f65 thru 0xeaf059d (0x9639 bytes) 2024-09-22T01:15:33.396 INFO:teuthology.orchestra.run.smithi096.stdout:5139 write 0xf07663 thru 0xf0bd22 (0x46c0 bytes) 2024-09-22T01:15:33.398 INFO:teuthology.orchestra.run.smithi096.stdout:5143 read 0x452eb17 thru 0x453cc8e (0xe178 bytes) 2024-09-22T01:15:33.399 INFO:teuthology.orchestra.run.smithi096.stdout:5144 write 0x1ea2dfe thru 0x1ea87f7 (0x59fa bytes) 2024-09-22T01:15:33.404 INFO:teuthology.orchestra.run.smithi096.stdout:5148 punch from 0x34c188d to 0x34ccef1, (0xb664 bytes) 2024-09-22T01:15:33.404 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.404 INFO:teuthology.orchestra.run.smithi096.stdout:5154 read 0x8c31d1c thru 0x8c3591e (0x3c03 bytes) 2024-09-22T01:15:33.404 INFO:teuthology.orchestra.run.smithi096.stdout:5156 read 0x8762af1 thru 0x876625b (0x376b bytes) 2024-09-22T01:15:33.412 INFO:teuthology.orchestra.run.smithi096.stdout:5160 trunc from 0xeaf059e to 0x9617de7 2024-09-22T01:15:33.420 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:33.417+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:33.426 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.437 INFO:teuthology.orchestra.run.smithi096.stdout:5163 write 0xc330a84 thru 0xc339c6f (0x91ec bytes) 2024-09-22T01:15:33.445 INFO:teuthology.orchestra.run.smithi096.stdout:5165 trunc from 0xc339c70 to 0xb1b721e 2024-09-22T01:15:33.454 INFO:teuthology.orchestra.run.smithi096.stdout:5166 read 0x7c71910 thru 0x7c80f06 (0xf5f7 bytes) 2024-09-22T01:15:33.454 INFO:teuthology.orchestra.run.smithi096.stdout:5167 punch from 0x3857215 to 0x3860363, (0x914e bytes) 2024-09-22T01:15:33.455 INFO:teuthology.orchestra.run.smithi096.stdout:5168 read 0x50be87 thru 0x50e41f (0x2599 bytes) 2024-09-22T01:15:33.455 INFO:teuthology.orchestra.run.smithi096.stdout:5170 read 0x18f4124 thru 0x18f6c5c (0x2b39 bytes) 2024-09-22T01:15:33.457 INFO:teuthology.orchestra.run.smithi096.stdout:5171 trunc from 0xb1b721e to 0xc3d0339 2024-09-22T01:15:33.464 INFO:teuthology.orchestra.run.smithi096.stdout:5173 read 0x3563c84 thru 0x356d0ae (0x942b bytes) 2024-09-22T01:15:33.465 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.465 INFO:teuthology.orchestra.run.smithi096.stdout:5177 writesame 0x8c981e9 thru 0x8ca7330 data_size 0x1c(0xf148 bytes) 2024-09-22T01:15:33.471 INFO:teuthology.orchestra.run.smithi096.stdout:5178 write 0xbe0ea3 thru 0xbe1909 (0xa67 bytes) 2024-09-22T01:15:33.474 INFO:teuthology.orchestra.run.smithi096.stdout:5179 write 0x859b4e0 thru 0x85a7488 (0xbfa9 bytes) 2024-09-22T01:15:33.479 INFO:teuthology.orchestra.run.smithi096.stdout:5180 write 0x71e3b08 thru 0x71ecd13 (0x920c bytes) 2024-09-22T01:15:33.483 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.483 INFO:teuthology.orchestra.run.smithi096.stdout:5182 read 0x6f09569 thru 0x6f0fa82 (0x651a bytes) 2024-09-22T01:15:33.485 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:33.485 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:33.495 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:33.495 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:33.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:33.495 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:33.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7226 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.179274903999996, 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015789705000000001, 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7226 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312840+0000", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312841+0000", 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312847+0000", 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-06 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312864+0000", 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7666999999999999e-05 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.312877+0000", 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.313e-05 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328191+0000", 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015313995 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328415+0000", 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000223835 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328630+0000", 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021450500000000001 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7227 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.179075256000004, 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016168779000000001, 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7227 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313040+0000", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313041+0000", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5969999999999999e-06 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313052+0000", 2024-09-22T01:15:33.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0966e-05 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313078+0000", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5429999999999999e-05 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313094+0000", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6784000000000001e-05 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328713+0000", 2024-09-22T01:15:33.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015618393 2024-09-22T01:15:33.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.328947+0000", 2024-09-22T01:15:33.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023425799999999999 2024-09-22T01:15:33.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329208+0000", 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000261351 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178996553000005, 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:33.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:33.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:33.510 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178936559999997, 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:33.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:33.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178864067000006, 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:33.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178795754999996, 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:33.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:33.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178737831000007, 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:33.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:33.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 77.178681941999997, 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:33.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 63.291737165000001, 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:33.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:33.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21740 3.4 3:206ad289:::rbd_object_map.11249c3a956.0000000000000010:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:33.537 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.596370184999998, 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016133075, 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21740 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:33.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895745+0000", 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895747+0000", 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.599e-06 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895761+0000", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3739e-05 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895798+0000", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6816000000000001e-05 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895819+0000", 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1336999999999999e-05 2024-09-22T01:15:33.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.895952+0000", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000132743 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.896168+0000", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000215606 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.897041+0000", 2024-09-22T01:15:33.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087390899999999997 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.906168+0000", 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0091266969999999996 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911648+0000", 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0054795340000000003 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911797+0000", 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014901399999999999 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.911878+0000", 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1081000000000005e-05 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.204245174, 2024-09-22T01:15:33.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23923 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.512390926000002, 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016008623, 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23923 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979724+0000", 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979738+0000", 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979749+0000", 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0402e-05 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979773+0000", 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4270999999999999e-05 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979790+0000", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7158000000000001e-05 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985606+0000", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0058163629999999997 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985828+0000", 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022168900000000001 2024-09-22T01:15:33.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986962+0000", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0011341159999999999 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995508+0000", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085453400000000002 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995535+0000", 2024-09-22T01:15:33.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7141e-05 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995654+0000", 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011901899999999999 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995733+0000", 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8872e-05 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23924 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.512272707000001, 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016204125, 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23924 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979842+0000", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979844+0000", 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5460000000000001e-06 2024-09-22T01:15:33.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979852+0000", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4959999999999996e-06 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979877+0000", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5151e-05 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.979894+0000", 2024-09-22T01:15:33.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6731999999999998e-05 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.987898+0000", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080042250000000002 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988147+0000", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024910299999999998 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.988873+0000", 2024-09-22T01:15:33.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00072595099999999996 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995758+0000", 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0068844780000000003 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995874+0000", 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115752 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995949+0000", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5286999999999999e-05 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.996046+0000", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7404000000000005e-05 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23925 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.509745766000002, 2024-09-22T01:15:33.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016337575, 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23925 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982369+0000", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982372+0000", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6460000000000002e-06 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982404+0000", 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2199999999999997e-05 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982438+0000", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.43e-05 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.982460+0000", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1195e-05 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990057+0000", 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075978499999999997 2024-09-22T01:15:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990230+0000", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000172957 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.990800+0000", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000569499 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.995776+0000", 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049762890000000001 2024-09-22T01:15:33.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998448+0000", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.002672167 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998614+0000", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000165871 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.998707+0000", 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2601000000000003e-05 2024-09-22T01:15:33.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.491299413, 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.565 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:33.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24023 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.331618425000002, 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016177754999999999, 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24023 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:33.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160497+0000", 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160498+0000", 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1370000000000001e-06 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160511+0000", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2999e-05 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160538+0000", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7396999999999999e-05 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160554+0000", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6311999999999999e-05 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167086+0000", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0065315729999999997 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167171+0000", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4667999999999996e-05 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.167606+0000", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00043582400000000002 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175772+0000", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081651970000000008 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176584+0000", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000812179 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176636+0000", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1733000000000001e-05 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176674+0000", 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.8736000000000003e-05 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24024 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.331532129000003, 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016189912000000001, 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24024 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160583+0000", 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160584+0000", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6300000000000001e-06 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160594+0000", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0850000000000003e-06 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160612+0000", 2024-09-22T01:15:33.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8808999999999999e-05 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.160628+0000", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6118000000000001e-05 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169454+0000", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0088255499999999997 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169530+0000", 2024-09-22T01:15:33.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6159e-05 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.169875+0000", 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034475800000000002 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.175807+0000", 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0059322869999999996 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176685+0000", 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00087780499999999995 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176737+0000", 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1745e-05 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.176773+0000", 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5966000000000001e-05 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 57.330902420000001, 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:33.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:33.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:114281 3.4 3:22d7e50b:::rbd_object_map.11245508ac2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:27.789949+0000", 2024-09-22T01:15:33.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.7021660980000002, 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015758706000000001, 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 114281 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789949+0000", 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789949+0000", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789950+0000", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2979999999999999e-06 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789956+0000", 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6710000000000004e-06 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789967+0000", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1338999999999999e-05 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789974+0000", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.365e-06 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790028+0000", 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4763999999999999e-05 2024-09-22T01:15:33.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790096+0000", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8010999999999996e-05 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790343+0000", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00024678800000000001 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805473+0000", 2024-09-22T01:15:33.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015130380000000001 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805577+0000", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010349300000000001 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805646+0000", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9469999999999997e-05 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805708+0000", 2024-09-22T01:15:33.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1126999999999999e-05 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:114282 3.4 3:22d7e50b:::rbd_object_map.11245508ac2b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:27.789989+0000", 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.7021255369999997, 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016013815000000001, 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 114282 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789989+0000", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789989+0000", 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:33.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789990+0000", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0200000000000003e-07 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789993+0000", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8279999999999999e-06 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.789997+0000", 2024-09-22T01:15:33.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5920000000000002e-06 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790002+0000", 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3120000000000004e-06 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790842+0000", 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084092399999999995 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:33.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.790958+0000", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000115153 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.791464+0000", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00050680300000000001 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805561+0000", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014096249999999999 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805935+0000", 2024-09-22T01:15:33.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037397100000000001 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.805974+0000", 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9620999999999999e-05 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:27.806003+0000", 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8958999999999998e-05 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:33.599 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:33.599 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:33.600 INFO:teuthology.orchestra.run.smithi096.stdout:5183 trunc from 0xc3d0339 to 0x1d26074 2024-09-22T01:15:33.600 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:33.600 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:33.601 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.601 INFO:teuthology.orchestra.run.smithi096.stdout:5185 punch from 0x1baf70c to 0x1bb48d3, (0x51c7 bytes) 2024-09-22T01:15:33.601 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:33.602 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:33.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:33.602 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:33.598+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:33.603 INFO:teuthology.orchestra.run.smithi096.stdout:5187 writesame 0xac54330 thru 0xac5e2fa data_size 0x1(0x9fcb bytes) 2024-09-22T01:15:33.603 INFO:teuthology.orchestra.run.smithi096.stdout:5188 trunc from 0xac5e2fb to 0xec4353 2024-09-22T01:15:33.603 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:33.603 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:33.604 INFO:teuthology.orchestra.run.smithi096.stdout:5189 trunc from 0xec4353 to 0x2653c1a 2024-09-22T01:15:33.604 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.604 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:33.604 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:33.605 INFO:teuthology.orchestra.run.smithi096.stdout:5194 write 0x2d26378 thru 0x2d2cec5 (0x6b4e bytes) 2024-09-22T01:15:33.605 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:33.605 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:33.610 INFO:teuthology.orchestra.run.smithi096.stdout:5195 write 0xe0ad913 thru 0xe0adb5e (0x24c bytes) 2024-09-22T01:15:33.615 INFO:teuthology.orchestra.run.smithi096.stdout:5197 read 0xa5e8d8d thru 0xa5f2e79 (0xa0ed bytes) 2024-09-22T01:15:33.615 INFO:teuthology.orchestra.run.smithi096.stdout:5198 writesame 0xa8c7c85 thru 0xa8d60ff data_size 0x61(0xe47b bytes) 2024-09-22T01:15:33.620 INFO:teuthology.orchestra.run.smithi096.stdout:5199 write 0x8b3c395 thru 0x8b41c06 (0x5872 bytes) 2024-09-22T01:15:33.624 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:33.624 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:33.626 INFO:teuthology.orchestra.run.smithi096.stdout:5200 writesame 0x1a1d581 thru 0x1a268f2 data_size 0x1b(0x9372 bytes) 2024-09-22T01:15:33.629 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.629 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:33.629 INFO:teuthology.orchestra.run.smithi096.stdout:5204 write 0x98ee2af thru 0x98fd582 (0xf2d4 bytes) 2024-09-22T01:15:33.635 INFO:teuthology.orchestra.run.smithi096.stdout:5205 writesame 0x9aeac72 thru 0x9aed876 data_size 0x1(0x2c05 bytes) 2024-09-22T01:15:33.645 INFO:teuthology.orchestra.run.smithi096.stdout:5206 write 0xdc2e7 thru 0xe88ff (0xc619 bytes) 2024-09-22T01:15:33.646 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:33.646 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:33.660 INFO:teuthology.orchestra.run.smithi096.stdout:5207 trunc from 0xe0adb5f to 0x7657e2d 2024-09-22T01:15:33.677 INFO:teuthology.orchestra.run.smithi096.stdout:5209 punch from 0x4b2ebb7 to 0x4b3ab08, (0xbf51 bytes) 2024-09-22T01:15:33.677 INFO:teuthology.orchestra.run.smithi096.stdout:5210 read 0x5db67e7 thru 0x5dbe47f (0x7c99 bytes) 2024-09-22T01:15:33.677 INFO:teuthology.orchestra.run.smithi096.stdout:5213 punch from 0x32a8508 to 0x32b3e50, (0xb948 bytes) 2024-09-22T01:15:33.677 INFO:teuthology.orchestra.run.smithi096.stdout:5214 read 0x5ecbbcf thru 0x5ed012d (0x455f bytes) 2024-09-22T01:15:33.682 INFO:teuthology.orchestra.run.smithi096.stdout:5215 trunc from 0x7657e2d to 0x3fe157d 2024-09-22T01:15:33.697 INFO:teuthology.orchestra.run.smithi096.stdout:5216 read 0x191e703 thru 0x1922a58 (0x4356 bytes) 2024-09-22T01:15:33.701 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:33.698+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:33.720 INFO:teuthology.orchestra.run.smithi096.stdout:5217 write 0xd11b481 thru 0xd125496 (0xa016 bytes) 2024-09-22T01:15:33.725 INFO:teuthology.orchestra.run.smithi096.stdout:5218 write 0xc1fd0c6 thru 0xc209a36 (0xc971 bytes) 2024-09-22T01:15:33.750 INFO:teuthology.orchestra.run.smithi096.stdout:5219 trunc from 0xd125497 to 0x1150cc6 2024-09-22T01:15:33.780 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:33.781 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:33.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:33.796 INFO:teuthology.orchestra.run.smithi096.stdout:5221 trunc from 0x1150cc6 to 0xdca6b30 2024-09-22T01:15:33.804 INFO:teuthology.orchestra.run.smithi096.stdout:5223 punch from 0xcbae80a to 0xcbaff21, (0x1717 bytes) 2024-09-22T01:15:33.805 INFO:teuthology.orchestra.run.smithi096.stdout:5224 writesame 0x3faf905 thru 0x3fbb7b0 data_size 0x2(0xbeac bytes) 2024-09-22T01:15:33.828 INFO:teuthology.orchestra.run.smithi096.stdout:5225 write 0x35b064c thru 0x35b97b0 (0x9165 bytes) 2024-09-22T01:15:33.838 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:33.838+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:33.842 INFO:teuthology.orchestra.run.smithi096.stdout:5226 write 0xc54d226 thru 0xc5576ab (0xa486 bytes) 2024-09-22T01:15:33.848 INFO:teuthology.orchestra.run.smithi096.stdout:5227 clone 42 order 22 su 16384 sc 12 2024-09-22T01:15:33.902 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:33.902+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:33.930 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:15:33.964 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:33.964 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:33.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:34.022 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:34.022+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:34.060 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone41 from 0xdca6b30 (overlap 0xa90263) to 0x764e78 2024-09-22T01:15:34.104 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:34.102+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:34.146 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:34.146 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:34.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:34.205 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:34.206+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:34.254 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-22T01:15:34.297 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:34.297 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:34.305 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:34.301+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:34.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:34.342 INFO:teuthology.orchestra.run.smithi096.stdout:5228 read 0x1544a84 thru 0x1545608 (0xb85 bytes) 2024-09-22T01:15:34.343 INFO:teuthology.orchestra.run.smithi096.stdout:5229 punch from 0xcd8e2b5 to 0xcd8ed21, (0xa6c bytes) 2024-09-22T01:15:34.351 INFO:teuthology.orchestra.run.smithi096.stdout:5230 trunc from 0xdca6b30 to 0xcdc8721 2024-09-22T01:15:34.407 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:34.401+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:34.418 INFO:teuthology.orchestra.run.smithi096.stdout:5231 writesame 0x87aa0b4 thru 0x87ab83e data_size 0x29(0x178b bytes) 2024-09-22T01:15:34.434 INFO:teuthology.orchestra.run.smithi096.stdout:5232 trunc from 0xcdc8721 to 0x52b2909 2024-09-22T01:15:34.490 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:34.490 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:34.506 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:34.508 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:34.506+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:34.564 INFO:teuthology.orchestra.run.smithi096.stdout:5233 trunc from 0x52b2909 to 0x7e03cb1 2024-09-22T01:15:34.572 INFO:teuthology.orchestra.run.smithi096.stdout:5234 punch from 0xb4f325 to 0xb57853, (0x852e bytes) 2024-09-22T01:15:34.572 INFO:teuthology.orchestra.run.smithi096.stdout:5235 write 0x595962a thru 0x5961227 (0x7bfe bytes) 2024-09-22T01:15:34.594 INFO:teuthology.orchestra.run.smithi096.stdout:5237 writesame 0xe0b8bd7 thru 0xe0c7a15 data_size 0x1(0xee3f bytes) 2024-09-22T01:15:34.594 INFO:teuthology.orchestra.run.smithi096.stdout:5238 write 0x230f30e thru 0x231efb8 (0xfcab bytes) 2024-09-22T01:15:34.609 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:34.606+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:34.709 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:34.709 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:34.709 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:34.706+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:34.723 INFO:teuthology.orchestra.run.smithi096.stdout:5239 write 0x7ff5c8d thru 0x7ffb1a1 (0x5515 bytes) 2024-09-22T01:15:34.724 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:34.745 INFO:teuthology.orchestra.run.smithi096.stdout:5240 trunc from 0xe0c7a16 to 0x390d042 2024-09-22T01:15:34.786 INFO:teuthology.orchestra.run.smithi096.stdout:5241 write 0xa23a02e thru 0xa23f050 (0x5023 bytes) 2024-09-22T01:15:34.792 INFO:teuthology.orchestra.run.smithi096.stdout:5242 write 0x4d66ed9 thru 0x4d734a7 (0xc5cf bytes) 2024-09-22T01:15:34.797 INFO:teuthology.orchestra.run.smithi096.stdout:5243 write 0x7b89598 thru 0x7b93192 (0x9bfb bytes) 2024-09-22T01:15:34.808 INFO:teuthology.orchestra.run.smithi096.stdout:5244 read 0x34c4c51 thru 0x34cf8bb (0xac6b bytes) 2024-09-22T01:15:34.810 INFO:teuthology.orchestra.run.smithi096.stdout:5245 punch from 0x2044f6e to 0x2052c26, (0xdcb8 bytes) 2024-09-22T01:15:34.810 INFO:teuthology.orchestra.run.smithi096.stdout:5246 read 0x60b5653 thru 0x60ba52d (0x4edb bytes) 2024-09-22T01:15:34.810 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:34.810 INFO:teuthology.orchestra.run.smithi096.stdout:5248 writesame 0x6ab0da8 thru 0x6abb7cc data_size 0x1(0xaa25 bytes) 2024-09-22T01:15:34.810 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:34.810+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:34.864 INFO:teuthology.orchestra.run.smithi096.stdout:5249 punch from 0x7333646 to 0x7337e17, (0x47d1 bytes) 2024-09-22T01:15:34.868 INFO:teuthology.orchestra.run.smithi096.stdout:5251 trunc from 0xa23f051 to 0xca3e876 2024-09-22T01:15:34.879 INFO:teuthology.orchestra.run.smithi096.stdout:5252 writesame 0x27f6690 thru 0x27f75db data_size 0x16(0xf4c bytes) 2024-09-22T01:15:34.880 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:34.880 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:34.887 INFO:teuthology.orchestra.run.smithi096.stdout:5253 trunc from 0xca3e876 to 0x9d1db31 2024-09-22T01:15:34.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:34.903 INFO:teuthology.orchestra.run.smithi096.stdout:5254 write 0x24db646 thru 0x24e88cb (0xd286 bytes) 2024-09-22T01:15:34.948 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:34.945+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:35.012 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:35.010+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:35.031 INFO:teuthology.orchestra.run.smithi096.stdout:5255 read 0x23395c8 thru 0x233b065 (0x1a9e bytes) 2024-09-22T01:15:35.031 INFO:teuthology.orchestra.run.smithi096.stdout:5256 write 0x8c0e5e6 thru 0x8c191d3 (0xabee bytes) 2024-09-22T01:15:35.036 INFO:teuthology.orchestra.run.smithi096.stdout:5258 read 0x66e776f thru 0x66eac99 (0x352b bytes) 2024-09-22T01:15:35.036 INFO:teuthology.orchestra.run.smithi096.stdout:5260 read 0x1f588b4 thru 0x1f5c016 (0x3763 bytes) 2024-09-22T01:15:35.037 INFO:teuthology.orchestra.run.smithi096.stdout:5261 punch from 0x3563ff5 to 0x3568467, (0x4472 bytes) 2024-09-22T01:15:35.051 INFO:teuthology.orchestra.run.smithi096.stdout:5262 write 0xdef400d thru 0xdef9d3b (0x5d2f bytes) 2024-09-22T01:15:35.058 INFO:teuthology.orchestra.run.smithi096.stdout:5263 read 0x3d36f0c thru 0x3d438eb (0xc9e0 bytes) 2024-09-22T01:15:35.059 INFO:teuthology.orchestra.run.smithi096.stdout:5265 punch from 0xdd26774 to 0xdd35381, (0xec0d bytes) 2024-09-22T01:15:35.061 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:35.061 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:35.067 INFO:teuthology.orchestra.run.smithi096.stdout:5266 writesame 0xea4e011 thru 0xea58861 data_size 0x35(0xa851 bytes) 2024-09-22T01:15:35.067 INFO:teuthology.orchestra.run.smithi096.stdout:5267 read 0x6e18fe6 thru 0x6e27263 (0xe27e bytes) 2024-09-22T01:15:35.068 INFO:teuthology.orchestra.run.smithi096.stdout:5268 punch from 0x74990be to 0x749bf48, (0x2e8a bytes) 2024-09-22T01:15:35.068 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:35.068 INFO:teuthology.orchestra.run.smithi096.stdout:5273 write 0xe336b6c thru 0xe33ba33 (0x4ec8 bytes) 2024-09-22T01:15:35.075 INFO:teuthology.orchestra.run.smithi096.stdout:5274 read 0x2e664d0 thru 0x2e72645 (0xc176 bytes) 2024-09-22T01:15:35.077 INFO:teuthology.orchestra.run.smithi096.stdout:5275 writesame 0xbdac6ab thru 0xbdb4383 data_size 0x1(0x7cd9 bytes) 2024-09-22T01:15:35.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:5277 read 0x3693272 thru 0x3698bfc (0x598b bytes) 2024-09-22T01:15:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:5280 read 0x804fc69 thru 0x8057bb1 (0x7f49 bytes) 2024-09-22T01:15:35.099 INFO:teuthology.orchestra.run.smithi096.stdout:5281 read 0x92e36bf thru 0x92ee6ab (0xafed bytes) 2024-09-22T01:15:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:5282 read 0x8fbd973 thru 0x8fc0b67 (0x31f5 bytes) 2024-09-22T01:15:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:5284 punch from 0xd6301e3 to 0xd635f9d, (0x5dba bytes) 2024-09-22T01:15:35.100 INFO:teuthology.orchestra.run.smithi096.stdout:5285 clone 43 order 20 su 65536 sc 12 2024-09-22T01:15:35.113 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:35.110+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:35.211 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:35.212 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:35.214 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:35.209+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:35.226 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:35.315 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:35.314+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:35.391 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:35.391 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:35.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:35.452 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:35.449+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:35.516 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:35.513+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:35.521 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:35.521 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:35.535 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:35.620 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:35.617+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:35.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:35.696 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:35.710 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:35.752 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:35.749+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:35.814 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:35.814 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:35.817 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:35.813+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:35.831 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:35.831 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:35.848 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:35.848 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:35.865 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:35.866 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:35.869 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:35.870 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:35.884 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:35.887 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:35.887 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:35.908 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:35.908 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:35.918 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:35.917+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:35.929 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:35.929 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:35.952 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:35.952 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:35.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:15:36.011 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:36.011 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:36.011 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:36.018 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:36.013+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:36.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:36.061 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone42 from 0xea58862 (overlap 0x390d042) to 0x5b4806 2024-09-22T01:15:36.119 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:36.113+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:36.214 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:36.214 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:36.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:36.214 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.19326375, 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:36.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:36.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:36.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:36.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.645699954000001, 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:36.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.634389288999998, 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:36.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.396038205, 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:36.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066162730999999, 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:36.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:36.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:36.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.06615171, 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066140885999999, 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:36.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:36.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:36.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066130007000002, 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:36.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:36.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066115994, 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066085069, 2024-09-22T01:15:36.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:36.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:36.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:36.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066059980999999, 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:36.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066035746000001, 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:36.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:36.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.066013730000002, 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:36.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:36.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065984221000001, 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:36.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:36.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:36.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:36.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065932535999998, 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:36.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:36.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065899906999999, 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:36.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065842529000001, 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:36.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065827465000002, 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:36.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:36.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:36.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065814541000002, 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:36.309 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:36.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:36.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:36.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.065802942000001, 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:36.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:36.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:36.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:36.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:36.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:36.318 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:36.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:36.320 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-22T01:15:36.321 INFO:teuthology.orchestra.run.smithi096.stdout:5287 trunc from 0xea58862 to 0x843f92c 2024-09-22T01:15:36.321 INFO:teuthology.orchestra.run.smithi096.stdout:5288 write 0x337413c thru 0x33769ef (0x28b4 bytes) 2024-09-22T01:15:36.321 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:36.313+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:36.333 INFO:teuthology.orchestra.run.smithi096.stdout:5290 write 0x85219b6 thru 0x8524121 (0x276c bytes) 2024-09-22T01:15:36.343 INFO:teuthology.orchestra.run.smithi096.stdout:5291 write 0x99dcf1a thru 0x99e518a (0x8271 bytes) 2024-09-22T01:15:36.349 INFO:teuthology.orchestra.run.smithi096.stdout:5292 read 0x790b37f thru 0x7912320 (0x6fa2 bytes) 2024-09-22T01:15:36.357 INFO:teuthology.orchestra.run.smithi096.stdout:5294 write 0xc1f517e thru 0xc203fab (0xee2e bytes) 2024-09-22T01:15:36.362 INFO:teuthology.orchestra.run.smithi096.stdout:5295 writesame 0x7b5e930 thru 0x7b61c22 data_size 0x1(0x32f3 bytes) 2024-09-22T01:15:36.388 INFO:teuthology.orchestra.run.smithi096.stdout:5297 trunc from 0xc203fac to 0x1cf1388 2024-09-22T01:15:36.420 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:36.417+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:36.460 INFO:teuthology.orchestra.run.smithi096.stdout:5299 writesame 0x7d8bcab thru 0x7d8c352 data_size 0xc(0x6a8 bytes) 2024-09-22T01:15:36.461 INFO:teuthology.orchestra.run.smithi096.stdout:5300 read 0x4a291c9 thru 0x4a2c5fa (0x3432 bytes) 2024-09-22T01:15:36.461 INFO:teuthology.orchestra.run.smithi096.stdout:5301 read 0xa6f676 thru 0xa7c0c8 (0xca53 bytes) 2024-09-22T01:15:36.467 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:36.468 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:36.468 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:36.468 INFO:teuthology.orchestra.run.smithi096.stdout:5302 trunc from 0x7d8c353 to 0xdbfafaf 2024-09-22T01:15:36.480 INFO:teuthology.orchestra.run.smithi096.stdout:5303 trunc from 0xdbfafaf to 0x8b6228c 2024-09-22T01:15:36.482 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:36.492 INFO:teuthology.orchestra.run.smithi096.stdout:5304 trunc from 0x8b6228c to 0x49bc0c6 2024-09-22T01:15:36.515 INFO:teuthology.orchestra.run.smithi096.stdout:5305 write 0xcf7ef31 thru 0xcf8bac0 (0xcb90 bytes) 2024-09-22T01:15:36.523 INFO:teuthology.orchestra.run.smithi096.stdout:5307 trunc from 0xcf8bac1 to 0xbd2a2aa 2024-09-22T01:15:36.526 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:36.525+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:36.538 INFO:teuthology.orchestra.run.smithi096.stdout:5308 read 0x55be30 thru 0x560512 (0x46e3 bytes) 2024-09-22T01:15:36.539 INFO:teuthology.orchestra.run.smithi096.stdout:5309 read 0x608a5af thru 0x609a0b5 (0xfb07 bytes) 2024-09-22T01:15:36.542 INFO:teuthology.orchestra.run.smithi096.stdout:5311 trunc from 0xbd2a2aa to 0xa485147 2024-09-22T01:15:36.553 INFO:teuthology.orchestra.run.smithi096.stdout:5313 writesame 0x80a3134 thru 0x80a60dc data_size 0x15(0x2fa9 bytes) 2024-09-22T01:15:36.559 INFO:teuthology.orchestra.run.smithi096.stdout:5314 punch from 0x453d209 to 0x454140e, (0x4205 bytes) 2024-09-22T01:15:36.559 INFO:teuthology.orchestra.run.smithi096.stdout:5315 writesame 0x41e59ec thru 0x41e99a9 data_size 0x29(0x3fbe bytes) 2024-09-22T01:15:36.565 INFO:teuthology.orchestra.run.smithi096.stdout:5316 read 0x6064ebe thru 0x60722f2 (0xd435 bytes) 2024-09-22T01:15:36.565 INFO:teuthology.orchestra.run.smithi096.stdout:5317 write 0x9310756 thru 0x931f363 (0xec0e bytes) 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:5318 read 0x98789d0 thru 0x9885652 (0xcc83 bytes) 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:5320 read 0x8d63eaf thru 0x8d667f1 (0x2943 bytes) 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:5322 punch from 0x4bf1eb1 to 0x4bfe602, (0xc751 bytes) 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:5325 read 0x97b9c9c thru 0x97c922d (0xf592 bytes) 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:36.571 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:36.578 INFO:teuthology.orchestra.run.smithi096.stdout:5328 trunc from 0xa485147 to 0x5f27e37 2024-09-22T01:15:36.615 INFO:teuthology.orchestra.run.smithi096.stdout:5329 write 0x8e97871 thru 0x8ea3f79 (0xc709 bytes) 2024-09-22T01:15:36.622 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:36.616+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:36.623 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:36.639 INFO:teuthology.orchestra.run.smithi096.stdout:5331 trunc from 0x8ea3f7a to 0x19d48f6 2024-09-22T01:15:36.651 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:36.651 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:36.651 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:36.651 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:36.651 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:36.668 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:36.669 INFO:teuthology.orchestra.run.smithi096.stdout:5332 read 0x11f3ab7 thru 0x11f60af (0x25f9 bytes) 2024-09-22T01:15:36.670 INFO:teuthology.orchestra.run.smithi096.stdout:5333 clone 44 order 18 su 65536 sc 9 2024-09-22T01:15:36.723 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:36.721+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433462312000003, 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:36.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433432042000007, 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:36.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433402469000001, 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:36.799 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:36.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:36.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:36.804 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433348378000005, 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:36.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.809 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433292534000003, 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.810 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:36.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:36.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433234643999995, 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:36.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:36.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:36.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433172911, 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:36.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433077448000006, 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.824 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:36.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.828 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.433015624999996, 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.829 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:36.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432924485000001, 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:36.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432862579000002, 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.839 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:36.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:36.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:36.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432802092000003, 2024-09-22T01:15:36.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:36.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:36.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432744622000001, 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:36.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:36.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432687341999994, 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:36.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:36.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432627818, 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.858 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:36.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.432574934000002, 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:36.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.423462372000003, 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:36.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:36.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.873 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.980203186000001, 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:36.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 31.959131406000001, 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:36.878 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:36.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:36.882 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 23.772813846999998, 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:36.883 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:36.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:36.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:36.888 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:36.889 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:36.890 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:36.891 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:36.888+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:36.943 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:36.943 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:15:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:15:36.991 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:36.989+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:37.066 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:37.066 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:37.066 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:37.076 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone43 from 0x19d48f6 (overlap 0x19d48f6) to 0x173f4dd 2024-09-22T01:15:37.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:37.126 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:37.125+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:37.191 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:37.188+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:37.250 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.250 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:37.250 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:37.250 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:37.250 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:37.265 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:37.268 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-22T01:15:37.284 INFO:teuthology.orchestra.run.smithi096.stdout:5335 read 0xd11eec thru 0xd13703 (0x1818 bytes) 2024-09-22T01:15:37.285 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:37.285 INFO:teuthology.orchestra.run.smithi096.stdout:5337 read 0x14c5989 thru 0x14cec12 (0x928a bytes) 2024-09-22T01:15:37.286 INFO:teuthology.orchestra.run.smithi096.stdout:5338 read 0x16ed60b thru 0x16f9124 (0xbb1a bytes) 2024-09-22T01:15:37.293 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:37.288+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:37.298 INFO:teuthology.orchestra.run.smithi096.stdout:5339 trunc from 0x19d48f6 to 0x9c43b50 2024-09-22T01:15:37.312 INFO:teuthology.orchestra.run.smithi096.stdout:5340 read 0x313046e thru 0x31347a8 (0x433b bytes) 2024-09-22T01:15:37.312 INFO:teuthology.orchestra.run.smithi096.stdout:5341 writesame 0x42ba1f3 thru 0x42c51a2 data_size 0x18(0xafb0 bytes) 2024-09-22T01:15:37.318 INFO:teuthology.orchestra.run.smithi096.stdout:5342 writesame 0x977bade thru 0x977d6ac data_size 0x15(0x1bcf bytes) 2024-09-22T01:15:37.335 INFO:teuthology.orchestra.run.smithi096.stdout:5343 writesame 0xca99899 thru 0xca9dafc data_size 0xe(0x4264 bytes) 2024-09-22T01:15:37.335 INFO:teuthology.orchestra.run.smithi096.stdout:5344 punch from 0x8b8831d to 0x8b95cb7, (0xd99a bytes) 2024-09-22T01:15:37.335 INFO:teuthology.orchestra.run.smithi096.stdout:5346 read 0xb0f3f5b thru 0xb0fa1ed (0x6293 bytes) 2024-09-22T01:15:37.336 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:37.336 INFO:teuthology.orchestra.run.smithi096.stdout:5348 read 0x42c0ae5 thru 0x42c8e93 (0x83af bytes) 2024-09-22T01:15:37.336 INFO:teuthology.orchestra.run.smithi096.stdout:5349 write 0x15054b5 thru 0x1508ba1 (0x36ed bytes) 2024-09-22T01:15:37.343 INFO:teuthology.orchestra.run.smithi096.stdout:5350 writesame 0x232efb2 thru 0x2334404 data_size 0x1(0x5453 bytes) 2024-09-22T01:15:37.361 INFO:teuthology.orchestra.run.smithi096.stdout:5351 writesame 0x4a1c80f thru 0x4a251e6 data_size 0x2c(0x89d8 bytes) 2024-09-22T01:15:37.368 INFO:teuthology.orchestra.run.smithi096.stdout:5352 write 0x8836f27 thru 0x8843b31 (0xcc0b bytes) 2024-09-22T01:15:37.376 INFO:teuthology.orchestra.run.smithi096.stdout:5353 read 0x6be5b78 thru 0x6bf5a67 (0xfef0 bytes) 2024-09-22T01:15:37.376 INFO:teuthology.orchestra.run.smithi096.stdout:5354 read 0x6620a24 thru 0x6623d13 (0x32f0 bytes) 2024-09-22T01:15:37.376 INFO:teuthology.orchestra.run.smithi096.stdout:5355 punch from 0x7b39d15 to 0x7b45a20, (0xbd0b bytes) 2024-09-22T01:15:37.387 INFO:teuthology.orchestra.run.smithi096.stdout:5356 write 0xda7c1c5 thru 0xda852c3 (0x90ff bytes) 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:37.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.245344212999999, 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:37.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:37.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:37.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.245244273000001, 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:37.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:37.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:37.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.245175381999999, 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:37.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:37.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.245099319000001, 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:37.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.245036531, 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244405597, 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244376458000001, 2024-09-22T01:15:37.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:37.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:37.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244347133000002, 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:37.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244316497, 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:37.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244280829000001, 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:37.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:37.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244251846000001, 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244217634000002, 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244148813999999, 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.244048299999999, 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243980145999998, 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243896415999998, 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243762537999999, 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243635689000001, 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243598733999999, 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.243547092, 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.472 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:37.473 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:37.388+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:37.474 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5357 writesame 0x218fddd thru 0x219dbb0 data_size 0x2(0xddd4 bytes) 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5360 writesame 0x2bef113 thru 0x2bf955b data_size 0x3(0xa449 bytes) 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5361 read 0x1c26ddd thru 0x1c32640 (0xb864 bytes) 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5362 write 0x3d51d5f thru 0x3d52fec (0x128e bytes) 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5363 read 0x45851aa thru 0x45859a2 (0x7f9 bytes) 2024-09-22T01:15:37.475 INFO:teuthology.orchestra.run.smithi096.stdout:5364 write 0x50aabf7 thru 0x50b0a31 (0x5e3b bytes) 2024-09-22T01:15:37.481 INFO:teuthology.orchestra.run.smithi096.stdout:5366 writesame 0x48568c4 thru 0x485a412 data_size 0x15(0x3b4f bytes) 2024-09-22T01:15:37.490 INFO:teuthology.orchestra.run.smithi096.stdout:5371 write 0x71f55f7 thru 0x71f89dc (0x33e6 bytes) 2024-09-22T01:15:37.500 INFO:teuthology.orchestra.run.smithi096.stdout:5372 read 0xbb0c546 thru 0xbb14d18 (0x87d3 bytes) 2024-09-22T01:15:37.500 INFO:teuthology.orchestra.run.smithi096.stdout:5373 write 0xef96b2 thru 0xf03628 (0x9f77 bytes) 2024-09-22T01:15:37.513 INFO:teuthology.orchestra.run.smithi096.stdout:5374 punch from 0x96b4e51 to 0x96bb4a2, (0x6651 bytes) 2024-09-22T01:15:37.513 INFO:teuthology.orchestra.run.smithi096.stdout:5376 writesame 0x3d5db6c thru 0x3d68fc3 data_size 0x74(0xb458 bytes) 2024-09-22T01:15:37.521 INFO:teuthology.orchestra.run.smithi096.stdout:5378 write 0xb9d817a thru 0xb9e0c45 (0x8acc bytes) 2024-09-22T01:15:37.522 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:37.521+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:37.531 INFO:teuthology.orchestra.run.smithi096.stdout:5379 write 0x3504a73 thru 0x350e720 (0x9cae bytes) 2024-09-22T01:15:37.538 INFO:teuthology.orchestra.run.smithi096.stdout:5381 write 0x38f6c01 thru 0x38fd55d (0x695d bytes) 2024-09-22T01:15:37.544 INFO:teuthology.orchestra.run.smithi096.stdout:5382 punch from 0x10ae9d8 to 0x10b6a45, (0x806d bytes) 2024-09-22T01:15:37.575 INFO:teuthology.orchestra.run.smithi096.stdout:5383 trunc from 0xda852c4 to 0x16d3391 2024-09-22T01:15:37.595 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:37.593+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:37.613 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.614 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:37.614 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:37.614 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:37.630 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:37.668 INFO:teuthology.orchestra.run.smithi096.stdout:5384 write 0x423cebf thru 0x424513e (0x8280 bytes) 2024-09-22T01:15:37.696 INFO:teuthology.orchestra.run.smithi096.stdout:5388 writesame 0x7058b8a thru 0x7065ae4 data_size 0x1(0xcf5b bytes) 2024-09-22T01:15:37.696 INFO:teuthology.orchestra.run.smithi096.stdout:5391 read 0x4b42315 thru 0x4b42804 (0x4f0 bytes) 2024-09-22T01:15:37.697 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:37.692+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:37.697 INFO:teuthology.orchestra.run.smithi096.stdout:5392 write 0x653dfa9 thru 0x65485cf (0xa627 bytes) 2024-09-22T01:15:37.705 INFO:teuthology.orchestra.run.smithi096.stdout:5395 read 0x2f588f4 thru 0x2f63367 (0xaa74 bytes) 2024-09-22T01:15:37.706 INFO:teuthology.orchestra.run.smithi096.stdout:5396 write 0x25bf442 thru 0x25cc974 (0xd533 bytes) 2024-09-22T01:15:37.718 INFO:teuthology.orchestra.run.smithi096.stdout:5397 writesame 0x11bd6d8 thru 0x11c563e data_size 0xb(0x7f67 bytes) 2024-09-22T01:15:37.737 INFO:teuthology.orchestra.run.smithi096.stdout:5398 write 0x2dcf7f7 thru 0x2dd6e6a (0x7674 bytes) 2024-09-22T01:15:37.747 INFO:teuthology.orchestra.run.smithi096.stdout:5399 punch from 0x4e78754 to 0x4e809c0, (0x826c bytes) 2024-09-22T01:15:37.747 INFO:teuthology.orchestra.run.smithi096.stdout:5403 write 0x3c30311 thru 0x3c3b373 (0xb063 bytes) 2024-09-22T01:15:37.754 INFO:teuthology.orchestra.run.smithi096.stdout:5405 punch from 0x2f00be0 to 0x2f0407d, (0x349d bytes) 2024-09-22T01:15:37.754 INFO:teuthology.orchestra.run.smithi096.stdout:5406 read 0x27bf895 thru 0x27cd731 (0xde9d bytes) 2024-09-22T01:15:37.754 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:37.772 INFO:teuthology.orchestra.run.smithi096.stdout:5408 write 0x8bfd3bc thru 0x8c00546 (0x318b bytes) 2024-09-22T01:15:37.783 INFO:teuthology.orchestra.run.smithi096.stdout:5409 read 0x6740938 thru 0x674eca9 (0xe372 bytes) 2024-09-22T01:15:37.784 INFO:teuthology.orchestra.run.smithi096.stdout:5411 writesame 0x21465cd thru 0x2151889 data_size 0x1(0xb2bd bytes) 2024-09-22T01:15:37.797 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:37.793+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:37.801 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.802 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:37.802 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:37.802 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:37.802 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:37.816 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:37.853 INFO:teuthology.orchestra.run.smithi096.stdout:5412 write 0x4eb0a4 thru 0x4f9e50 (0xedad bytes) 2024-09-22T01:15:37.873 INFO:teuthology.orchestra.run.smithi096.stdout:5413 clone 45 order 21 su 65536 sc 7 2024-09-22T01:15:37.902 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:37.901+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:37.959 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:15:37.974 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:37.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:15:37.989 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:37.989 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:37.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:37.989 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672348744000004, 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:37.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:37.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:37.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672288750999996, 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:37.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:37.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672216258000006, 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:37.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:37.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672147945999995, 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.003 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:38.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:38.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:38.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672090022000006, 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.007 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 81.672034132999997, 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.785089356, 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.697597365000007, 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 61.984651604, 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 61.824254611000001, 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122050 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2850816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.580327893, 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016781720999999999, 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122050 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405140+0000", 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405142+0000", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1220000000000002e-06 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405148+0000", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2110000000000003e-06 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405153+0000", 2024-09-22T01:15:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421546+0000", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016393557 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421632+0000", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5611000000000004e-05 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421921+0000", 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028914000000000001 2024-09-22T01:15:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122051 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2916352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5803073109999999, 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017012590000000001, 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122051 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.82e-07 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405163+0000", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2390000000000001e-06 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405169+0000", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6050000000000003e-06 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405191+0000", 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1625999999999999e-05 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421958+0000", 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016766873000000002 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422046+0000", 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8041000000000006e-05 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422172+0000", 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012672399999999999 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122052 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2981888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.580192738, 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017140268, 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122052 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405276+0000", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.874e-06 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405290+0000", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3325e-05 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405319+0000", 2024-09-22T01:15:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9791999999999999e-05 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405338+0000", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9004999999999998e-05 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422212+0000", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016874005000000001 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422295+0000", 2024-09-22T01:15:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2891000000000003e-05 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422415+0000", 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119376 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122230 3.4 3:235bf1fd:::rbd_data.1124ddce6057.0000000000000002:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5767101939999999, 2024-09-22T01:15:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01649316, 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122230 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408758+0000", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408763+0000", 2024-09-22T01:15:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1760000000000001e-06 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408790+0000", 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6855e-05 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408807+0000", 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7152000000000001e-05 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425125+0000", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016317941999999998 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425176+0000", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1263e-05 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425250+0000", 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3621999999999994e-05 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122241 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2801664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.576095118, 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016601057999999998, 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122241 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409373+0000", 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2270000000000001e-06 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409379+0000", 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409396+0000", 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6985000000000001e-05 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409408+0000", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2819e-05 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425570+0000", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016161983000000001 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425671+0000", 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010096 2024-09-22T01:15:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425973+0000", 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030178699999999998 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122242 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5760431079999999, 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01685275, 2024-09-22T01:15:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122242 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409425+0000", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409430+0000", 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599999999999998e-06 2024-09-22T01:15:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409447+0000", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409459+0000", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2672e-05 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426007+0000", 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016547671 2024-09-22T01:15:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426065+0000", 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7652000000000003e-05 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426277+0000", 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021208300000000001 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122243 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2932736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5759921180000001, 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017069536, 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122243 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409476+0000", 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:15:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409481+0000", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409498+0000", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409511+0000", 2024-09-22T01:15:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426325+0000", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016814030000000001 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426439+0000", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011437700000000001 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426545+0000", 2024-09-22T01:15:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105628 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122244 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.575940662, 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017170747, 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122244 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409527+0000", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409532+0000", 2024-09-22T01:15:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8130000000000003e-06 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409549+0000", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6575999999999999e-05 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409562+0000", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2772000000000001e-05 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426577+0000", 2024-09-22T01:15:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017014945 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426631+0000", 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4537999999999999e-05 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426697+0000", 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6075999999999996e-05 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122245 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.5758895050000001, 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017515171, 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122245 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409579+0000", 2024-09-22T01:15:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.026e-06 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409583+0000", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6829999999999999e-06 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409646+0000", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2803000000000002e-05 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409662+0000", 2024-09-22T01:15:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5418999999999999e-05 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426742+0000", 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017080346 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426830+0000", 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8527000000000001e-05 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427093+0000", 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000262367 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122246 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.575770047, 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017546235, 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122246 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409698+0000", 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.283e-06 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409713+0000", 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4839e-05 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409729+0000", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5931999999999999e-05 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409796+0000", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7149000000000004e-05 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427115+0000", 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017319068 2024-09-22T01:15:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427166+0000", 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0784e-05 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427243+0000", 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7180000000000003e-05 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:38.079 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:38.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:38.081 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:38.076+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:38.084 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone44 from 0x8c00547 (overlap 0x16d3391) to 0x0 2024-09-22T01:15:38.139 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:38.139 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:38.173 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:38.173 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:38.181 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:38.177+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:38.199 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:38.200 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:38.239 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:38.282 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:38.276+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:38.288 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:38.288 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:38.293 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:38.293 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:38.293 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:38.293 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:38.307 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:38.310 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:38.331 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:38.331 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:38.354 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:38.355 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:38.396 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:38.393+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:38.418 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-22T01:15:38.478 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:38.479 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:38.483 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:38.476+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:38.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:38.503 INFO:teuthology.orchestra.run.smithi096.stdout:5414 write 0x4292eff thru 0x429e5f5 (0xb6f7 bytes) 2024-09-22T01:15:38.520 INFO:teuthology.orchestra.run.smithi096.stdout:5415 trunc from 0x8c00547 to 0xbf2623e 2024-09-22T01:15:38.528 INFO:teuthology.orchestra.run.smithi096.stdout:5417 read 0xb7b9abb thru 0xb7bcdf7 (0x333d bytes) 2024-09-22T01:15:38.537 INFO:teuthology.orchestra.run.smithi096.stdout:5421 writesame 0xcbd50b3 thru 0xcbdb93b data_size 0x1(0x6889 bytes) 2024-09-22T01:15:38.537 INFO:teuthology.orchestra.run.smithi096.stdout:5422 punch from 0x15059ca to 0x1510fc1, (0xb5f7 bytes) 2024-09-22T01:15:38.556 INFO:teuthology.orchestra.run.smithi096.stdout:5423 trunc from 0xcbdb93c to 0x2d7513 2024-09-22T01:15:38.584 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:38.580+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:38.618 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:38.618 INFO:teuthology.orchestra.run.smithi096.stdout:5426 clone 46 order 20 su 4096 sc 10 2024-09-22T01:15:38.674 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:38.674 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:38.686 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:38.680+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:38.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:38.787 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:38.785+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:38.818 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:38.818 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:38.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:38.887 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:38.880+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:38.957 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:38.957 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:38.970 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:15:38.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:38.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:15:38.988 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:38.985+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:38.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:15:39.089 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:39.085+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:39.127 INFO:teuthology.orchestra.run.smithi096.stdout:leaving image image_client.0-clone45 intact 2024-09-22T01:15:39.166 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.166 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:39.232 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:39.228+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:39.242 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-22T01:15:39.245 INFO:teuthology.orchestra.run.smithi096.stdout:5427 read 0x2ad7a2 thru 0x2bc37e (0xebdd bytes) 2024-09-22T01:15:39.250 INFO:teuthology.orchestra.run.smithi096.stdout:5428 clone 47 order 24 su 8388608 sc 12 2024-09-22T01:15:39.291 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:39.284+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:39.312 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.312 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:39.412 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:39.408+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:39.482 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.483 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.493 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:39.488+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:39.496 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:39.594 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:39.593+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:39.656 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.657 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:39.695 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:39.693+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.786 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.796 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:39.792+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:39.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:39.897 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:39.892+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:39.966 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:39.966 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:15:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:15:39.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:39.997 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:39.993+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:40.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:15:40.098 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:40.097+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:40.118 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone46 from 0x2d7513 (overlap 0x2d7513) to 0x0 2024-09-22T01:15:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:40.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:40.198 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:40.197+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:40.299 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:40.297+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:40.315 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:40.315 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:40.329 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:40.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:40.383 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-22T01:15:40.393 INFO:teuthology.orchestra.run.smithi096.stdout:5429 read 0x2bcea6 thru 0x2c9a71 (0xcbcc bytes) 2024-09-22T01:15:40.400 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:40.396+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:40.419 INFO:teuthology.orchestra.run.smithi096.stdout:5430 write 0xaa0c99a thru 0xaa10994 (0x3ffb bytes) 2024-09-22T01:15:40.424 INFO:teuthology.orchestra.run.smithi096.stdout:5431 write 0x340e7b9 thru 0x34128cc (0x4114 bytes) 2024-09-22T01:15:40.430 INFO:teuthology.orchestra.run.smithi096.stdout:5432 writesame 0x98a89e thru 0x995d97 data_size 0x71(0xb4fa bytes) 2024-09-22T01:15:40.440 INFO:teuthology.orchestra.run.smithi096.stdout:5433 write 0xb570364 thru 0xb57cea3 (0xcb40 bytes) 2024-09-22T01:15:40.446 INFO:teuthology.orchestra.run.smithi096.stdout:5436 read 0x39be241 thru 0x39c20d0 (0x3e90 bytes) 2024-09-22T01:15:40.446 INFO:teuthology.orchestra.run.smithi096.stdout:5437 punch from 0x5a65e66 to 0x5a6f389, (0x9523 bytes) 2024-09-22T01:15:40.451 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:40.451 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:40.455 INFO:teuthology.orchestra.run.smithi096.stdout:5438 trunc from 0xb57cea4 to 0x35a99eb 2024-09-22T01:15:40.476 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:40.484 INFO:teuthology.orchestra.run.smithi096.stdout:5439 read 0x26f73cc thru 0x2705d34 (0xe969 bytes) 2024-09-22T01:15:40.486 INFO:teuthology.orchestra.run.smithi096.stdout:5440 trunc from 0x35a99eb to 0x4eb2b19 2024-09-22T01:15:40.492 INFO:teuthology.orchestra.run.smithi096.stdout:5441 read 0x18f2dfb thru 0x18fcfb1 (0xa1b7 bytes) 2024-09-22T01:15:40.492 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.501 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:40.497+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:40.510 INFO:teuthology.orchestra.run.smithi096.stdout:5444 write 0xe41abf8 thru 0xe4205ee (0x59f7 bytes) 2024-09-22T01:15:40.513 INFO:teuthology.orchestra.run.smithi096.stdout:5445 writesame 0x2f146fe thru 0x2f17fbf data_size 0x5(0x38c2 bytes) 2024-09-22T01:15:40.521 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.521 INFO:teuthology.orchestra.run.smithi096.stdout:5448 read 0xa64dc86 thru 0xa655065 (0x73e0 bytes) 2024-09-22T01:15:40.521 INFO:teuthology.orchestra.run.smithi096.stdout:5449 read 0x5af8429 thru 0x5b015ac (0x9184 bytes) 2024-09-22T01:15:40.521 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.521 INFO:teuthology.orchestra.run.smithi096.stdout:5451 writesame 0x2df65ee thru 0x2e05edc data_size 0x1(0xf8ef bytes) 2024-09-22T01:15:40.539 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:40.540 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:40.556 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:40.557 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:40.560 INFO:teuthology.orchestra.run.smithi096.stdout:5452 writesame 0x72d100b thru 0x72dadae data_size 0xc(0x9da4 bytes) 2024-09-22T01:15:40.574 INFO:teuthology.orchestra.run.smithi096.stdout:5453 trunc from 0xe4205ef to 0x8dad329 2024-09-22T01:15:40.575 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:40.575 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:40.593 INFO:teuthology.orchestra.run.smithi096.stdout:5454 read 0x50ecc6f thru 0x50fb5d7 (0xe969 bytes) 2024-09-22T01:15:40.594 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:40.594 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:40.601 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:40.597+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:40.603 INFO:teuthology.orchestra.run.smithi096.stdout:5455 trunc from 0x8dad329 to 0x1bcdc6 2024-09-22T01:15:40.619 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:40.619 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:40.620 INFO:teuthology.orchestra.run.smithi096.stdout:5456 read 0x8c093 thru 0x8e859 (0x27c7 bytes) 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.602970786, 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:15:40.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.055406989999994, 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:15:40.631 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:15:40.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4213.0:4 3.11 3:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-22T01:15:40.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.044096324999998, 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064163768999999996, 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "reached pg", 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4213", 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/2332359600", 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578728+0000", 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578730+0000", 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9e-06 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578742+0000", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1143e-05 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578771+0000", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9147999999999999e-05 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578785+0000", 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3993e-05 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578912+0000", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012710900000000001 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.578927+0000", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5556000000000001e-05 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642785+0000", 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063858122000000003 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:39.642892+0000", 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106798 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:97822 3.14 3:2ad5d157:::rbd_data.1124ccc9ab83.0000000000000061:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.805745240999997, 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065543755999999995, 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 97822 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817080+0000", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817087+0000", 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5649999999999998e-06 2024-09-22T01:15:40.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817104+0000", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7297999999999999e-05 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817113+0000", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5520000000000005e-06 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817169+0000", 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6047000000000001e-05 2024-09-22T01:15:40.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.817309+0000", 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013967100000000001 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:04.882623+0000", 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065314709999999998 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103542 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 483328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475869766999999, 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056707401999999997, 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103542 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146955+0000", 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146956+0000", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.097e-06 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146960+0000", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.455e-06 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146963+0000", 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8320000000000002e-06 2024-09-22T01:15:40.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203561+0000", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056598835 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203614+0000", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2162e-05 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203662+0000", 2024-09-22T01:15:40.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8797000000000001e-05 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103543 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 499712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475858746, 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056915764000000001, 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103543 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146966+0000", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2399999999999999e-07 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146967+0000", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.062e-06 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146971+0000", 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3519999999999998e-06 2024-09-22T01:15:40.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146973+0000", 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7149999999999998e-06 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203703+0000", 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056729235000000003 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203796+0000", 2024-09-22T01:15:40.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3651000000000002e-05 2024-09-22T01:15:40.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203882+0000", 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5525000000000005e-05 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103544 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 516096~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475847922, 2024-09-22T01:15:40.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057045223999999999, 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103544 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146977+0000", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2499999999999999e-07 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146978+0000", 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.006e-06 2024-09-22T01:15:40.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146981+0000", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2969999999999999e-06 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146984+0000", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9500000000000001e-06 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203924+0000", 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056939602999999998 2024-09-22T01:15:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.203977+0000", 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2873e-05 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204022+0000", 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5269999999999999e-05 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103549 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 468992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475837042999999, 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057252098000000001, 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103549 2024-09-22T01:15:40.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146988+0000", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2600000000000001e-07 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146992+0000", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1019999999999999e-06 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146995+0000", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3610000000000001e-06 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.146998+0000", 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.88e-06 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204060+0000", 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057062024000000003 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204154+0000", 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3445999999999999e-05 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204240+0000", 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6058999999999997e-05 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103550 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 485376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475823030000001, 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057375170000000003, 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103550 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147002+0000", 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4600000000000001e-07 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147003+0000", 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.029e-06 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147008+0000", 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5709999999999999e-06 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147013+0000", 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9080000000000003e-06 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204282+0000", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057269243999999997 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204332+0000", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0537000000000001e-05 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204377+0000", 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4635000000000002e-05 2024-09-22T01:15:40.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103551 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 501760~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475792105, 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057577807000000002, 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103551 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147033+0000", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.37e-07 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147035+0000", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2050000000000001e-06 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147042+0000", 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8979999999999999e-06 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147049+0000", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6810000000000001e-06 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204418+0000", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057368721999999997 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204513+0000", 2024-09-22T01:15:40.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5025000000000006e-05 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204610+0000", 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7738999999999994e-05 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103552 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 518144~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475767016999999, 2024-09-22T01:15:40.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701369000000002, 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103552 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147058+0000", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0600000000000001e-07 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147060+0000", 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0849999999999999e-06 2024-09-22T01:15:40.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147067+0000", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4339999999999997e-06 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147074+0000", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1369999999999996e-06 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204654+0000", 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057580582999999998 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204715+0000", 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0868000000000001e-05 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204759+0000", 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3955999999999997e-05 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.680 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103557 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 471040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475742782000001, 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057897915000000001, 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103557 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147082+0000", 2024-09-22T01:15:40.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147084+0000", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.086e-06 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147091+0000", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5919999999999997e-06 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147097+0000", 2024-09-22T01:15:40.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.857e-06 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204799+0000", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057701745999999998 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204892+0000", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3171999999999995e-05 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.204980+0000", 2024-09-22T01:15:40.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8106e-05 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103558 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475720765999998, 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058007211000000003, 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103558 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147104+0000", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7300000000000002e-07 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147113+0000", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3319999999999992e-06 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147119+0000", 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147125+0000", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5289999999999999e-06 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205020+0000", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057895279000000001 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205069+0000", 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8946999999999997e-05 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205111+0000", 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2372e-05 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103559 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 503808~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475691257000001, 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058190102, 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103559 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147134+0000", 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9700000000000002e-07 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147136+0000", 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1969999999999999e-06 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147154+0000", 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.77e-05 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147159+0000", 2024-09-22T01:15:40.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4519999999999996e-06 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205149+0000", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057990141000000002 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205242+0000", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2176000000000007e-05 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205324+0000", 2024-09-22T01:15:40.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2039000000000005e-05 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103560 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 520192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475639571999999, 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058269824999999997, 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103560 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147185+0000", 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147186+0000", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147189+0000", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4750000000000002e-06 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147198+0000", 2024-09-22T01:15:40.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0690000000000008e-06 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147205+0000", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6089999999999997e-06 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205363+0000", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058158287000000003 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205414+0000", 2024-09-22T01:15:40.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0352999999999997e-05 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205455+0000", 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1352e-05 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103565 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 473088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475606942999999, 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058462165000000003, 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103565 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147218+0000", 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147221+0000", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147233+0000", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2288e-05 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147244+0000", 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0684e-05 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205496+0000", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058251678000000001 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205592+0000", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5878999999999995e-05 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205680+0000", 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103566 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475549565000001, 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058536245000000001, 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103566 2024-09-22T01:15:40.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147275+0000", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147276+0000", 2024-09-22T01:15:40.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147277+0000", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147282+0000", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7620000000000003e-06 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147285+0000", 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3510000000000002e-06 2024-09-22T01:15:40.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205721+0000", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058435960000000002 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205769+0000", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7410000000000002e-05 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205811+0000", 2024-09-22T01:15:40.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2623000000000002e-05 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103567 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 505856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475534500999998, 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058736154999999998, 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103567 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147290+0000", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147291+0000", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147292+0000", 2024-09-22T01:15:40.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2109999999999999e-06 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147295+0000", 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5200000000000002e-06 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147298+0000", 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8770000000000002e-06 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205850+0000", 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058551404000000001 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.205941+0000", 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1352000000000002e-05 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206026+0000", 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5568000000000005e-05 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103568 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 522240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475521576999999, 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058859537000000003, 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103568 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147303+0000", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147304+0000", 2024-09-22T01:15:40.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147305+0000", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.175e-06 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147308+0000", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4659999999999999e-06 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147311+0000", 2024-09-22T01:15:40.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9620000000000002e-06 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206073+0000", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058761612999999997 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206118+0000", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5729999999999998e-05 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206163+0000", 2024-09-22T01:15:40.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4252000000000003e-05 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:103569 3.1d 3:bf0d8091:::rbd_data.1124772a770f.0000000000000001:1f [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.475509978000002, 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.059093761000000002, 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:40.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 103569 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147315+0000", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4400000000000001e-07 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147316+0000", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1060000000000001e-06 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147320+0000", 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4410000000000002e-06 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.147322+0000", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6309999999999999e-06 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206193+0000", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058870608999999997 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206287+0000", 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4469000000000006e-05 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:40.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:09.206409+0000", 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121261 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:40.720 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:40.721 INFO:teuthology.orchestra.run.smithi096.stdout:5457 write 0x140348a thru 0x140f619 (0xc190 bytes) 2024-09-22T01:15:40.721 INFO:teuthology.orchestra.run.smithi096.stdout:5458 read 0x12eabf3 thru 0x12f8c51 (0xe05f bytes) 2024-09-22T01:15:40.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:40.722 INFO:teuthology.orchestra.run.smithi096.stdout:5459 write 0x7e8077c thru 0x7e8de3e (0xd6c3 bytes) 2024-09-22T01:15:40.723 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:40.723 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:40.723 INFO:teuthology.orchestra.run.smithi096.stdout:5461 write 0xc929ac0 thru 0xc9387ab (0xecec bytes) 2024-09-22T01:15:40.723 INFO:teuthology.orchestra.run.smithi096.stdout:5462 read 0xaa30d81 thru 0xaa3a92f (0x9baf bytes) 2024-09-22T01:15:40.724 INFO:teuthology.orchestra.run.smithi096.stdout:5463 write 0x7e91fa6 thru 0x7e9b221 (0x927c bytes) 2024-09-22T01:15:40.724 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:40.724 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5464 read 0x4e1f30f thru 0x4e25ebc (0x6bae bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5465 trunc from 0xc9387ac to 0xca92dc5 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5466 write 0xcf89bd thru 0xcfbc5c (0x32a0 bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5467 punch from 0x29c6620 to 0x29cb7dd, (0x51bd bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5468 read 0x3cb6dfc thru 0x3cc254c (0xb751 bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5469 read 0xbd5fdf6 thru 0xbd6a579 (0xa784 bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5470 read 0x1df4d71 thru 0x1dfa4e5 (0x5775 bytes) 2024-09-22T01:15:40.725 INFO:teuthology.orchestra.run.smithi096.stdout:5471 writesame 0xbe70c0a thru 0xbe71f03 data_size 0x2(0x12fa bytes) 2024-09-22T01:15:40.726 INFO:teuthology.orchestra.run.smithi096.stdout:5473 writesame 0xad6592f thru 0xad67e44 data_size 0x2(0x2516 bytes) 2024-09-22T01:15:40.726 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:40.726 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:40.726 INFO:teuthology.orchestra.run.smithi096.stdout:5474 trunc from 0xca92dc5 to 0xd7de4cf 2024-09-22T01:15:40.726 INFO:teuthology.orchestra.run.smithi096.stdout:5475 read 0xc7cc510 thru 0xc7cf7d4 (0x32c5 bytes) 2024-09-22T01:15:40.727 INFO:teuthology.orchestra.run.smithi096.stdout:5476 writesame 0xdccd265 thru 0xdcce67f data_size 0x1(0x141b bytes) 2024-09-22T01:15:40.728 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.728 INFO:teuthology.orchestra.run.smithi096.stdout:5478 read 0x78f0fda thru 0x7900cb9 (0xfce0 bytes) 2024-09-22T01:15:40.728 INFO:teuthology.orchestra.run.smithi096.stdout:5480 writesame 0xc003cf0 thru 0xc012455 data_size 0x1b(0xe766 bytes) 2024-09-22T01:15:40.728 INFO:teuthology.orchestra.run.smithi096.stdout:5482 writesame 0x45bed6d thru 0x45c44b8 data_size 0x4a(0x574c bytes) 2024-09-22T01:15:40.728 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:40.717+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:40.729 INFO:teuthology.orchestra.run.smithi096.stdout:5483 read 0xb8c3b72 thru 0xb8ce271 (0xa700 bytes) 2024-09-22T01:15:40.729 INFO:teuthology.orchestra.run.smithi096.stdout:5484 write 0x27204b4 thru 0x27271e1 (0x6d2e bytes) 2024-09-22T01:15:40.735 INFO:teuthology.orchestra.run.smithi096.stdout:5486 read 0x9c781c0 thru 0x9c7ba0d (0x384e bytes) 2024-09-22T01:15:40.735 INFO:teuthology.orchestra.run.smithi096.stdout:5488 read 0x4c6e505 thru 0x4c70ead (0x29a9 bytes) 2024-09-22T01:15:40.735 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.735 INFO:teuthology.orchestra.run.smithi096.stdout:5490 writesame 0xcd9bd1 thru 0xce8043 data_size 0x1(0xe473 bytes) 2024-09-22T01:15:40.784 INFO:teuthology.orchestra.run.smithi096.stdout:5491 read 0x34a4950 thru 0x34afbfc (0xb2ad bytes) 2024-09-22T01:15:40.784 INFO:teuthology.orchestra.run.smithi096.stdout:5494 write 0xa577bdd thru 0xa586bb3 (0xefd7 bytes) 2024-09-22T01:15:40.788 INFO:teuthology.orchestra.run.smithi096.stdout:5496 read 0x1eaf2ac thru 0x1eb91aa (0x9eff bytes) 2024-09-22T01:15:40.789 INFO:teuthology.orchestra.run.smithi096.stdout:5497 write 0xb7c4bb4 thru 0xb7d4223 (0xf670 bytes) 2024-09-22T01:15:40.794 INFO:teuthology.orchestra.run.smithi096.stdout:5499 write 0xa0fa2f6 thru 0xa103c39 (0x9944 bytes) 2024-09-22T01:15:40.796 INFO:teuthology.orchestra.run.smithi096.stdout:5500 write 0x19be251 thru 0x19c95c2 (0xb372 bytes) 2024-09-22T01:15:40.799 INFO:teuthology.orchestra.run.smithi096.stdout:5501 flatten 2024-09-22T01:15:40.808 INFO:teuthology.orchestra.run.smithi096.stdout:5502 read 0x352aabc thru 0x3531bff (0x7144 bytes) 2024-09-22T01:15:40.809 INFO:teuthology.orchestra.run.smithi096.stdout:5503 write 0x635f862 thru 0x63652a4 (0x5a43 bytes) 2024-09-22T01:15:40.811 INFO:teuthology.orchestra.run.smithi096.stdout:5505 write 0xb82dedb thru 0xb83c837 (0xe95d bytes) 2024-09-22T01:15:40.815 INFO:teuthology.orchestra.run.smithi096.stdout:5506 read 0x5417bc4 thru 0x542272b (0xab68 bytes) 2024-09-22T01:15:40.815 INFO:teuthology.orchestra.run.smithi096.stdout:5509 writesame 0x255d872 thru 0x2566f39 data_size 0x64(0x96c8 bytes) 2024-09-22T01:15:40.819 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.819 INFO:teuthology.orchestra.run.smithi096.stdout:5512 read 0xd22b454 thru 0xd23aecf (0xfa7c bytes) 2024-09-22T01:15:40.819 INFO:teuthology.orchestra.run.smithi096.stdout:5513 read 0xd167d05 thru 0xd16ee0f (0x710b bytes) 2024-09-22T01:15:40.819 INFO:teuthology.orchestra.run.smithi096.stdout:5514 read 0xa8ed0ed thru 0xa8f0ef0 (0x3e04 bytes) 2024-09-22T01:15:40.819 INFO:teuthology.orchestra.run.smithi096.stdout:5515 write 0x3755f17 thru 0x37638a1 (0xd98b bytes) 2024-09-22T01:15:40.821 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:40.816+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:40.824 INFO:teuthology.orchestra.run.smithi096.stdout:5517 read 0x67e6f4 thru 0x67ed05 (0x612 bytes) 2024-09-22T01:15:40.839 INFO:teuthology.orchestra.run.smithi096.stdout:5518 trunc from 0xdcce680 to 0x504d608 2024-09-22T01:15:40.864 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:40.864 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:40.864 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:40.864 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:40.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:40.906 INFO:teuthology.orchestra.run.smithi096.stdout:5520 write 0xb41bf96 thru 0xb41ed9d (0x2e08 bytes) 2024-09-22T01:15:40.910 INFO:teuthology.orchestra.run.smithi096.stdout:5522 writesame 0x47bb61e thru 0x47c6eb0 data_size 0x1(0xb893 bytes) 2024-09-22T01:15:40.926 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:40.925+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:40.975 INFO:teuthology.orchestra.run.smithi096.stdout:5523 write 0xc51df98 thru 0xc5218fd (0x3966 bytes) 2024-09-22T01:15:40.983 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:40.983 INFO:teuthology.orchestra.run.smithi096.stdout:5525 writesame 0xa2a58bf thru 0xa2af409 data_size 0x1(0x9b4b bytes) 2024-09-22T01:15:40.987 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:15:41.000 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:41.000 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:41.001 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:41.001 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:41.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:41.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:15:41.019 INFO:teuthology.orchestra.run.smithi096.stdout:5526 write 0xc8c756 thru 0xc965c4 (0x9e6f bytes) 2024-09-22T01:15:41.023 INFO:teuthology.orchestra.run.smithi096.stdout:5527 punch from 0x3bf614d to 0x3bf7817, (0x16ca bytes) 2024-09-22T01:15:41.023 INFO:teuthology.orchestra.run.smithi096.stdout:5528 writesame 0xa44a07f thru 0xa4509f8 data_size 0x17(0x697a bytes) 2024-09-22T01:15:41.039 INFO:teuthology.orchestra.run.smithi096.stdout:5529 trunc from 0xc5218fe to 0x29f6346 2024-09-22T01:15:41.058 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:41.057+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:41.081 INFO:teuthology.orchestra.run.smithi096.stdout:5530 trunc from 0x29f6346 to 0xdaab470 2024-09-22T01:15:41.087 INFO:teuthology.orchestra.run.smithi096.stdout:5532 read 0x9b4abda thru 0x9b4cb8d (0x1fb4 bytes) 2024-09-22T01:15:41.087 INFO:teuthology.orchestra.run.smithi096.stdout:5533 writesame 0x7e7c838 thru 0x7e83b64 data_size 0x1(0x732d bytes) 2024-09-22T01:15:41.105 INFO:teuthology.orchestra.run.smithi096.stdout:5536 writesame 0xa0e42bd thru 0xa0e54ab data_size 0x1(0x11ef bytes) 2024-09-22T01:15:41.117 INFO:teuthology.orchestra.run.smithi096.stdout:5537 punch from 0x24a74b9 to 0x24b1061, (0x9ba8 bytes) 2024-09-22T01:15:41.117 INFO:teuthology.orchestra.run.smithi096.stdout:5538 read 0xd7b494b thru 0xd7bdbff (0x92b5 bytes) 2024-09-22T01:15:41.117 INFO:teuthology.orchestra.run.smithi096.stdout:5540 write 0x579bddb thru 0x57aa4dc (0xe702 bytes) 2024-09-22T01:15:41.122 INFO:teuthology.orchestra.run.smithi096.stdout:5541 read 0x9ac2616 thru 0x9acc313 (0x9cfe bytes) 2024-09-22T01:15:41.122 INFO:teuthology.orchestra.run.smithi096.stdout:5542 punch from 0x3177fce to 0x3178da4, (0xdd6 bytes) 2024-09-22T01:15:41.122 INFO:teuthology.orchestra.run.smithi096.stdout:5543 write 0x6f01ed6 thru 0x6f058ab (0x39d6 bytes) 2024-09-22T01:15:41.124 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:41.121+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:41.125 INFO:teuthology.orchestra.run.smithi096.stdout:5544 read 0x6aa9128 thru 0x6aab040 (0x1f19 bytes) 2024-09-22T01:15:41.125 INFO:teuthology.orchestra.run.smithi096.stdout:5545 write 0x132c84e thru 0x1331f62 (0x5715 bytes) 2024-09-22T01:15:41.128 INFO:teuthology.orchestra.run.smithi096.stdout:5546 write 0x3c0c514 thru 0x3c12293 (0x5d80 bytes) 2024-09-22T01:15:41.133 INFO:teuthology.orchestra.run.smithi096.stdout:5548 punch from 0x5876d08 to 0x587ba93, (0x4d8b bytes) 2024-09-22T01:15:41.133 INFO:teuthology.orchestra.run.smithi096.stdout:5549 punch from 0x9448a06 to 0x9454fab, (0xc5a5 bytes) 2024-09-22T01:15:41.133 INFO:teuthology.orchestra.run.smithi096.stdout:5550 write 0x62bb4ac thru 0x62bba3b (0x590 bytes) 2024-09-22T01:15:41.134 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:41.134 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:41.135 INFO:teuthology.orchestra.run.smithi096.stdout:5555 write 0x1c075a7 thru 0x1c09c2d (0x2687 bytes) 2024-09-22T01:15:41.138 INFO:teuthology.orchestra.run.smithi096.stdout:5556 read 0x8e8d7e1 thru 0x8e8f5c2 (0x1de2 bytes) 2024-09-22T01:15:41.138 INFO:teuthology.orchestra.run.smithi096.stdout:5557 punch from 0x20b5da1 to 0x20c29a3, (0xcc02 bytes) 2024-09-22T01:15:41.138 INFO:teuthology.orchestra.run.smithi096.stdout:5560 punch from 0xb838699 to 0xb844359, (0xbcc0 bytes) 2024-09-22T01:15:41.138 INFO:teuthology.orchestra.run.smithi096.stdout:5561 writesame 0x3c31139 thru 0x3c37916 data_size 0x5(0x67de bytes) 2024-09-22T01:15:41.144 INFO:teuthology.orchestra.run.smithi096.stdout:5562 write 0xccd2059 thru 0xccdecb8 (0xcc60 bytes) 2024-09-22T01:15:41.149 INFO:teuthology.orchestra.run.smithi096.stdout:5563 writesame 0xc47f66f thru 0xc481b5e data_size 0x18(0x24f0 bytes) 2024-09-22T01:15:41.154 INFO:teuthology.orchestra.run.smithi096.stdout:5564 punch from 0x516ca40 to 0x5177244, (0xa804 bytes) 2024-09-22T01:15:41.154 INFO:teuthology.orchestra.run.smithi096.stdout:5565 trunc from 0xdaab470 to 0xd77e73d 2024-09-22T01:15:41.161 INFO:teuthology.orchestra.run.smithi096.stdout:5568 read 0x1105410 thru 0x11091ff (0x3df0 bytes) 2024-09-22T01:15:41.161 INFO:teuthology.orchestra.run.smithi096.stdout:5569 write 0x3143c70 thru 0x314c2c0 (0x8651 bytes) 2024-09-22T01:15:41.166 INFO:teuthology.orchestra.run.smithi096.stdout:5571 write 0x1847ddd thru 0x1854f93 (0xd1b7 bytes) 2024-09-22T01:15:41.169 INFO:teuthology.orchestra.run.smithi096.stdout:5572 read 0x4c78594 thru 0x4c7d612 (0x507f bytes) 2024-09-22T01:15:41.170 INFO:teuthology.orchestra.run.smithi096.stdout:5573 punch from 0x2270b1e to 0x2279099, (0x857b bytes) 2024-09-22T01:15:41.170 INFO:teuthology.orchestra.run.smithi096.stdout:5575 writesame 0xca6a9a thru 0xca7df5 data_size 0x2a(0x135c bytes) 2024-09-22T01:15:41.171 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.824215112999994, 2024-09-22T01:15:41.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:41.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:41.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.824184842999998, 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.824155270000006, 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:41.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:41.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:41.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.824101178999996, 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.197 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.200 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:41.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:41.202 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.824045334999994, 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:41.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:41.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823987445, 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.208 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:41.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:41.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823925712000005, 2024-09-22T01:15:41.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:41.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823830248999997, 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:41.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:41.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823768426000001, 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:41.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.228 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823677286000006, 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.229 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:41.233 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823615380000007, 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.234 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:41.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:41.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.238 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823554892999994, 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:41.239 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:41.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:41.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.242 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:41.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.244 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823497423000006, 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.245 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.247 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:41.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.249 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823440142999999, 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.250 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.252 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.253 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:41.254 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823380619000005, 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.255 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.256 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:41.257 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.258 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.259 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.823327735000007, 2024-09-22T01:15:41.260 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.261 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.262 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:41.263 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.264 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.265 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.814215172999994, 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.266 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.267 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:41.268 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.269 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:41.270 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:41.271 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.370955987000002, 2024-09-22T01:15:41.272 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.273 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:41.274 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.275 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:41.276 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:41.277 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 36.349884207000002, 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.279 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:41.280 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.281 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 28.163566648, 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.283 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:41.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:41.287 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.288 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.288 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.288 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.288 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:41.290 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5579 trunc from 0xd77e73d to 0x1706029 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5580 write 0xb24f72d thru 0xb25f6dd (0xffb1 bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5581 read 0x9f514af thru 0x9f5d3a0 (0xbef2 bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5586 read 0x1825bfe thru 0x182b971 (0x5d74 bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5587 trunc from 0xb25f6de to 0x17f837d 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5588 read 0xc39c44 thru 0xc3ec62 (0x501f bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5589 writesame 0x290da93 thru 0x29178fa data_size 0x4a(0x9e68 bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5590 writesame 0x52fc229 thru 0x5309652 data_size 0xd(0xd42a bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5591 write 0x42e36fe thru 0x42e6e3a (0x373d bytes) 2024-09-22T01:15:41.292 INFO:teuthology.orchestra.run.smithi096.stdout:5592 punch from 0x4c07b8f to 0x4c0a84b, (0x2cbc bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5593 read 0x2378fa9 thru 0x237dbb4 (0x4c0c bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5594 read 0x4a59f06 thru 0x4a60e99 (0x6f94 bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5596 read 0x47c2d11 thru 0x47ce7f1 (0xbae1 bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5598 write 0x40ed86e thru 0x40f504f (0x77e2 bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5599 read 0x384f0f2 thru 0x385a8c7 (0xb7d6 bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:5600 read 0x4de91bc thru 0x4df31d4 (0xa019 bytes) 2024-09-22T01:15:41.293 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:41.293 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:41.285+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:41.298 INFO:teuthology.orchestra.run.smithi096.stdout:5603 trunc from 0x5309653 to 0xacb553b 2024-09-22T01:15:41.306 INFO:teuthology.orchestra.run.smithi096.stdout:5604 writesame 0x6288669 thru 0x628cdd5 data_size 0x45(0x476d bytes) 2024-09-22T01:15:41.309 INFO:teuthology.orchestra.run.smithi096.stdout:5605 punch from 0x906d5f3 to 0x907758a, (0x9f97 bytes) 2024-09-22T01:15:41.309 INFO:teuthology.orchestra.run.smithi096.stdout:5607 write 0x3c2b644 thru 0x3c2f8c2 (0x427f bytes) 2024-09-22T01:15:41.313 INFO:teuthology.orchestra.run.smithi096.stdout:5608 write 0x9220020 thru 0x922ca32 (0xca13 bytes) 2024-09-22T01:15:41.319 INFO:teuthology.orchestra.run.smithi096.stdout:5609 punch from 0x75f4af0 to 0x75f683f, (0x1d4f bytes) 2024-09-22T01:15:41.320 INFO:teuthology.orchestra.run.smithi096.stdout:5611 trunc from 0xacb553b to 0xbcb1a31 2024-09-22T01:15:41.336 INFO:teuthology.orchestra.run.smithi096.stdout:5612 write 0xce72f01 thru 0xce78102 (0x5202 bytes) 2024-09-22T01:15:41.342 INFO:teuthology.orchestra.run.smithi096.stdout:5613 punch from 0x93ef98a to 0x93f7335, (0x79ab bytes) 2024-09-22T01:15:41.342 INFO:teuthology.orchestra.run.smithi096.stdout:5615 clone 48 order 18 su 2048 sc 8 2024-09-22T01:15:41.390 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:41.389+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:41.466 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:41.466 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:41.467 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:41.467 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:41.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:41.522 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:41.521+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:41.592 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:41.584+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:41.642 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:41.642 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:41.642 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:41.642 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:41.642 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:41.656 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:41.692 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:41.692+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:41.781 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.633622166999999, 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.633522227, 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.633453336000002, 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:41.791 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:41.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.633377273000001, 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:41.796 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:41.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:41.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.800 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.633314485, 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.801 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:41.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:41.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.804 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632683551, 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.805 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632654412000001, 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:41.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:41.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:41.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632625087000001, 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:41.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632594451000003, 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.821 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632558783, 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:41.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:41.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.6325298, 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:41.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:41.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632495587999998, 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:41.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:41.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632426768000002, 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.837 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632326253999999, 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:41.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:41.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632258100000001, 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:41.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:41.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.845 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632174370000001, 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.846 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.849 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.632040492000002, 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.850 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:41.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:41.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.631913642999997, 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.854 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:41.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:41.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.631876687999998, 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.858 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:41.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:41.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.631825046000003, 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:41.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:41.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:41.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:41.866 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:41.868 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:41.868 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:41.865+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:41.969 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:41.968+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:42.005 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:15:42.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:15:42.035 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:15:42.041 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:42.041 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:42.041 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:42.056 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:42.102 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:42.101+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:42.171 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:42.164+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:42.172 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.172 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:42.172 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:42.172 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:42.172 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:42.186 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:42.253 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-22T01:15:42.255 INFO:teuthology.orchestra.run.smithi096.stdout:5617 read 0x8d477a7 thru 0x8d52929 (0xb183 bytes) 2024-09-22T01:15:42.262 INFO:teuthology.orchestra.run.smithi096.stdout:5618 trunc from 0xce78103 to 0x8f256ec 2024-09-22T01:15:42.272 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:42.268+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:42.300 INFO:teuthology.orchestra.run.smithi096.stdout:5619 write 0xbbb0e10 thru 0xbbb868e (0x787f bytes) 2024-09-22T01:15:42.312 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.327 INFO:teuthology.orchestra.run.smithi096.stdout:5621 trunc from 0xbbb868f to 0x130f90d 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.048215333000002, 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:42.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.048155339999994, 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.048082847000003, 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.373 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:42.377 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.048014534999993, 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.047956611000004, 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.382 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:42.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:42.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:42.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 86.047900721999994, 2024-09-22T01:15:42.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:42.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:42.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 72.160955944999998, 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:42.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:42.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:42.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.073463954000005, 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:42.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:42.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.360518193000004, 2024-09-22T01:15:42.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:42.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:42.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:42.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 66.200121199999998, 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:42.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:42.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:42.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122050 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2850816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9561944819999999, 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016781720999999999, 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122050 2024-09-22T01:15:42.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405140+0000", 2024-09-22T01:15:42.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405142+0000", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1220000000000002e-06 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405148+0000", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2110000000000003e-06 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405153+0000", 2024-09-22T01:15:42.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421546+0000", 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016393557 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421632+0000", 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5611000000000004e-05 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421921+0000", 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028914000000000001 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122051 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2916352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9561738999999996, 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017012590000000001, 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.415 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122051 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.82e-07 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405163+0000", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2390000000000001e-06 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405169+0000", 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6050000000000003e-06 2024-09-22T01:15:42.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405191+0000", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1625999999999999e-05 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421958+0000", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016766873000000002 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422046+0000", 2024-09-22T01:15:42.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8041000000000006e-05 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422172+0000", 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012672399999999999 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122052 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2981888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9560593270000002, 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017140268, 2024-09-22T01:15:42.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122052 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405276+0000", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.874e-06 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405290+0000", 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3325e-05 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405319+0000", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9791999999999999e-05 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405338+0000", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9004999999999998e-05 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422212+0000", 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016874005000000001 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422295+0000", 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2891000000000003e-05 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422415+0000", 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119376 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122230 3.4 3:235bf1fd:::rbd_data.1124ddce6057.0000000000000002:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.423 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9525767829999996, 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01649316, 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122230 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408758+0000", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408763+0000", 2024-09-22T01:15:42.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1760000000000001e-06 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408790+0000", 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6855e-05 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408807+0000", 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7152000000000001e-05 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425125+0000", 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016317941999999998 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425176+0000", 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1263e-05 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425250+0000", 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3621999999999994e-05 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122241 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2801664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9519617069999997, 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016601057999999998, 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122241 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409373+0000", 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2270000000000001e-06 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409379+0000", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409396+0000", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6985000000000001e-05 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409408+0000", 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2819e-05 2024-09-22T01:15:42.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425570+0000", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016161983000000001 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425671+0000", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010096 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425973+0000", 2024-09-22T01:15:42.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030178699999999998 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122242 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9519096969999996, 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01685275, 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122242 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:42.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409425+0000", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409430+0000", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599999999999998e-06 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409447+0000", 2024-09-22T01:15:42.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409459+0000", 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2672e-05 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426007+0000", 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016547671 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426065+0000", 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7652000000000003e-05 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426277+0000", 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021208300000000001 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122243 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2932736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9518587069999995, 2024-09-22T01:15:42.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017069536, 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122243 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409476+0000", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409481+0000", 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409498+0000", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409511+0000", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426325+0000", 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016814030000000001 2024-09-22T01:15:42.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426439+0000", 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011437700000000001 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426545+0000", 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105628 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122244 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.951807251, 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017170747, 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122244 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:42.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409527+0000", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409532+0000", 2024-09-22T01:15:42.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8130000000000003e-06 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409549+0000", 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6575999999999999e-05 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409562+0000", 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2772000000000001e-05 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426577+0000", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017014945 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426631+0000", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4537999999999999e-05 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426697+0000", 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6075999999999996e-05 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122245 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9517560940000003, 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017515171, 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122245 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409579+0000", 2024-09-22T01:15:42.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.026e-06 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409583+0000", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6829999999999999e-06 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409646+0000", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2803000000000002e-05 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409662+0000", 2024-09-22T01:15:42.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5418999999999999e-05 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426742+0000", 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017080346 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426830+0000", 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8527000000000001e-05 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427093+0000", 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000262367 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122246 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.9516366359999999, 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017546235, 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:42.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122246 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409698+0000", 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.283e-06 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409713+0000", 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4839e-05 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409729+0000", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5931999999999999e-05 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409796+0000", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7149000000000004e-05 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427115+0000", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017319068 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427166+0000", 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0784e-05 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427243+0000", 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7180000000000003e-05 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.453 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:42.455 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:42.455 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:42.453+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:42.544 INFO:teuthology.orchestra.run.smithi096.stdout:5623 write 0x376bffd thru 0x377451f (0x8523 bytes) 2024-09-22T01:15:42.556 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:42.552+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:42.577 INFO:teuthology.orchestra.run.smithi096.stdout:5625 write 0xc06e2a0 thru 0xc079bc1 (0xb922 bytes) 2024-09-22T01:15:42.590 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.601 INFO:teuthology.orchestra.run.smithi096.stdout:5628 trunc from 0xc079bc2 to 0x301ed84 2024-09-22T01:15:42.624 INFO:teuthology.orchestra.run.smithi096.stdout:5629 read 0x12bf632 thru 0x12c9149 (0x9b18 bytes) 2024-09-22T01:15:42.636 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.636 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:42.636 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:42.636 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:42.642 INFO:teuthology.orchestra.run.smithi096.stdout:5631 writesame 0x7cb4c01 thru 0x7cb9ec1 data_size 0x13(0x52c1 bytes) 2024-09-22T01:15:42.642 INFO:teuthology.orchestra.run.smithi096.stdout:5632 punch from 0x41b60d5 to 0x41b768c, (0x15b7 bytes) 2024-09-22T01:15:42.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:42.661 INFO:teuthology.orchestra.run.smithi096.stdout:5634 write 0xeab65a1 thru 0xeac4190 (0xdbf0 bytes) 2024-09-22T01:15:42.687 INFO:teuthology.orchestra.run.smithi096.stdout:5636 write 0x8e09564 thru 0x8e0abf0 (0x168d bytes) 2024-09-22T01:15:42.697 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.697 INFO:teuthology.orchestra.run.smithi096.stdout:5638 read 0x18dbe9b thru 0x18ea80b (0xe971 bytes) 2024-09-22T01:15:42.698 INFO:teuthology.orchestra.run.smithi096.stdout:5639 write 0x5189f2b thru 0x5197413 (0xd4e9 bytes) 2024-09-22T01:15:42.702 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:42.701+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:42.719 INFO:teuthology.orchestra.run.smithi096.stdout:5641 writesame 0xb5cea49 thru 0xb5d114b data_size 0x1(0x2703 bytes) 2024-09-22T01:15:42.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:42.737 INFO:teuthology.orchestra.run.smithi096.stdout:5643 punch from 0x8c73dc2 to 0x8c77965, (0x3ba3 bytes) 2024-09-22T01:15:42.737 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.737 INFO:teuthology.orchestra.run.smithi096.stdout:5645 read 0xbaa0c7b thru 0xbaa9323 (0x86a9 bytes) 2024-09-22T01:15:42.753 INFO:teuthology.orchestra.run.smithi096.stdout:5646 trunc from 0xeac4191 to 0x56f3576 2024-09-22T01:15:42.758 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:42.753+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:42.785 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.785 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:42.785 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:42.785 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:42.785 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:42.794 INFO:teuthology.orchestra.run.smithi096.stdout:5648 write 0x93ab812 thru 0x93baa40 (0xf22f bytes) 2024-09-22T01:15:42.800 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:42.818 INFO:teuthology.orchestra.run.smithi096.stdout:5649 writesame 0xbd0211a thru 0xbd0530d data_size 0x2e(0x31f4 bytes) 2024-09-22T01:15:42.819 INFO:teuthology.orchestra.run.smithi096.stdout:5650 punch from 0x46de03 to 0x47ae43, (0xd040 bytes) 2024-09-22T01:15:42.827 INFO:teuthology.orchestra.run.smithi096.stdout:5651 writesame 0xcf9cf4e thru 0xcf9efb8 data_size 0x1(0x206b bytes) 2024-09-22T01:15:42.827 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.836 INFO:teuthology.orchestra.run.smithi096.stdout:5653 write 0xe0fffa2 thru 0xe10649e (0x64fd bytes) 2024-09-22T01:15:42.853 INFO:teuthology.orchestra.run.smithi096.stdout:5654 write 0x5800322 thru 0x580bd65 (0xba44 bytes) 2024-09-22T01:15:42.858 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:42.852+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:42.874 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.874 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:42.874 INFO:teuthology.orchestra.run.smithi096.stdout:5659 writesame 0xa964867 thru 0xa973364 data_size 0x7(0xeafe bytes) 2024-09-22T01:15:42.900 INFO:teuthology.orchestra.run.smithi096.stdout:5660 trunc from 0xe10649f to 0x763842c 2024-09-22T01:15:42.902 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:42.903 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:42.923 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:42.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:42.924 INFO:teuthology.orchestra.run.smithi096.stdout:5661 read 0x4bbc2b5 thru 0x4bc7f24 (0xbc70 bytes) 2024-09-22T01:15:42.924 INFO:teuthology.orchestra.run.smithi096.stdout:5662 write 0x3a3a1e6 thru 0x3a3f86e (0x5689 bytes) 2024-09-22T01:15:42.947 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:42.947 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:42.952 INFO:teuthology.orchestra.run.smithi096.stdout:5663 write 0xc21911c thru 0xc228525 (0xf40a bytes) 2024-09-22T01:15:42.959 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:42.952+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:42.965 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:42.965 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:42.969 INFO:teuthology.orchestra.run.smithi096.stdout:5664 write 0x533a6ca thru 0x533ac0f (0x546 bytes) 2024-09-22T01:15:42.985 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:42.985 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:42.985 INFO:teuthology.orchestra.run.smithi096.stdout:5665 write 0xc97c9be thru 0xc98c214 (0xf857 bytes) 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10114 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.758576255999998, 2024-09-22T01:15:42.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015674179, 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10114 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227411+0000", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0699999999999996e-07 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227416+0000", 2024-09-22T01:15:42.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6029999999999998e-06 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227426+0000", 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7999999999999993e-06 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227433+0000", 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4630000000000004e-06 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231642+0000", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0042085480000000003 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.231861+0000", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021953399999999999 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.232570+0000", 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000709069 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242859+0000", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.010288230000000001 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242886+0000", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7282999999999999e-05 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.242971+0000", 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4790000000000006e-05 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243085+0000", 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000114152 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10115 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.758501426999999, 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016022966999999999, 2024-09-22T01:15:42.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10115 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227485+0000", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227486+0000", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0070000000000001e-06 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227491+0000", 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9069999999999998e-06 2024-09-22T01:15:42.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227503+0000", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1919999999999999e-05 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227511+0000", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0800000000000006e-06 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233845+0000", 2024-09-22T01:15:42.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0063334309999999996 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.233983+0000", 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000138169 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.235101+0000", 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001118312 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:42.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243122+0000", 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0080206070000000008 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243224+0000", 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101813 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243374+0000", 2024-09-22T01:15:42.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015015199999999999 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243508+0000", 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013456899999999999 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10116 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.758463402999993, 2024-09-22T01:15:43.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016433579, 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10116 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227523+0000", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227524+0000", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3e-07 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227528+0000", 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1529999999999999e-06 2024-09-22T01:15:43.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227552+0000", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3353000000000002e-05 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227562+0000", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.9129999999999998e-06 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236160+0000", 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085981740000000001 2024-09-22T01:15:43.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.236398+0000", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000237832 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.237133+0000", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00073499100000000005 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243139+0000", 2024-09-22T01:15:43.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060064389999999997 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243691+0000", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00055177000000000004 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243834+0000", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000143165 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243957+0000", 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012305899999999999 2024-09-22T01:15:43.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10117 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.758414181000006, 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016733880999999999, 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10117 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227573+0000", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2399999999999997e-07 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227580+0000", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0630000000000002e-06 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227591+0000", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1800000000000001e-05 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227599+0000", 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4499999999999998e-06 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238735+0000", 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011136551 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.238930+0000", 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019506800000000001 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.239573+0000", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064228899999999999 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.243541+0000", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0039684289999999999 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244056+0000", 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051518600000000005 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244198+0000", 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000141989 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244307+0000", 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010833200000000001 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:10118 3.b 3:d08934ea:::rbd_object_map.1124891c5fc4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.758377834000001, 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017146431, 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 10118 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227609+0000", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227610+0000", 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999997e-07 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227615+0000", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7400000000000004e-06 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227624+0000", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6160000000000008e-06 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.227631+0000", 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.007e-06 2024-09-22T01:15:43.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241016+0000", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013385360000000001 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241150+0000", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013350600000000001 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 6", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.241748+0000", 2024-09-22T01:15:43.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00059788199999999999 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244343+0000", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0025950359999999998 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244546+0000", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000203346 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244689+0000", 2024-09-22T01:15:43.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000142662 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:22.244755+0000", 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6575999999999994e-05 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21593 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 2621440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 72.099504944000003, 2024-09-22T01:15:43.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016303371000000001, 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21593 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886482+0000", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886484+0000", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.68e-06 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886492+0000", 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7120000000000006e-06 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886516+0000", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.389e-05 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.886532+0000", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6096e-05 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894119+0000", 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0075864749999999996 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.894354+0000", 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00023493799999999999 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902785+0000", 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0084315799999999993 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:21720 3.b 3:d0c78087:::rbd_data.11247e2569dd.0000000000000008:f [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 72.097189896000003, 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014493541, 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 21720 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.022 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888797+0000", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888798+0000", 2024-09-22T01:15:43.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3200000000000001e-06 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888805+0000", 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0520000000000004e-06 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888825+0000", 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9375e-05 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.888839+0000", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4622e-05 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902231+0000", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013391534 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.902946+0000", 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00071528900000000003 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.903291+0000", 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034434899999999998 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22366 3.c e48/24 3:30763d85:::rbd_data.1124dfca90b7.0000000000000004:head v 48'110, mlcod=47'107)", 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.694022317000005, 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015766703, 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.026 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.291965+0000", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6400000000000001e-07 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297240+0000", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0052751050000000004 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297266+0000", 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5466e-05 2024-09-22T01:15:43.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297280+0000", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4431e-05 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297388+0000", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107908 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.297414+0000", 2024-09-22T01:15:43.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5386999999999999e-05 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307091+0000", 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0096768759999999992 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307731+0000", 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00064076599999999997 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.030 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23880 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7340032~11415] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.017021236000005, 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015088861, 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23880 2024-09-22T01:15:43.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968966+0000", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968967+0000", 2024-09-22T01:15:43.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.22e-06 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968972+0000", 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3399999999999997e-06 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.968990+0000", 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8074999999999999e-05 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969004+0000", 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4134999999999999e-05 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983425+0000", 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014420511 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.983679+0000", 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000254145 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984055+0000", 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037543500000000003 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23887 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:43.035 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.016963860999994, 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015753017000000001, 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23887 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:43.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969023+0000", 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969024+0000", 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.133e-06 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969030+0000", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6729999999999997e-06 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969048+0000", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7835000000000001e-05 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969062+0000", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4252e-05 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984192+0000", 2024-09-22T01:15:43.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015129777000000001 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984468+0000", 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00027595700000000001 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984776+0000", 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030839000000000002 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23903 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 5505024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.016895151, 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016334753, 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23903 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969092+0000", 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969093+0000", 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2759999999999999e-06 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969099+0000", 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.353e-06 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969118+0000", 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9082000000000001e-05 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969133+0000", 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4591e-05 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.984896+0000", 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015762504 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985140+0000", 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000244684 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985426+0000", 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028626299999999998 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23906 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.016833207000005, 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016979752000000001, 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23906 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969154+0000", 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969155+0000", 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2160000000000001e-06 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969161+0000", 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6370000000000004e-06 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969180+0000", 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9198000000000002e-05 2024-09-22T01:15:43.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969198+0000", 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8408000000000001e-05 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985570+0000", 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016371918999999999 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.985840+0000", 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000269638 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986133+0000", 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029373600000000001 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23909 3.b 3:d288fb51:::rbd_data.1124dfca90b7.0000000000000009:11 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.016769240000002, 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017382827, 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23909 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969218+0000", 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969219+0000", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969224+0000", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969246+0000", 2024-09-22T01:15:43.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1475e-05 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.969260+0000", 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4378e-05 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986232+0000", 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016971763000000001 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986427+0000", 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019486500000000001 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:35.986600+0000", 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017356000000000001 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:29266 3.6 e61/24 3:66f55c8a:::rbd_data.11242d507bd4.0000000000000008:head v 61'201, mlcod=59'198)", 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:43.054 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 59.160674100000001, 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014763744000000001, 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825313+0000", 2024-09-22T01:15:43.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825314+0000", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825324+0000", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5929999999999995e-06 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825337+0000", 2024-09-22T01:15:43.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3417999999999999e-05 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825348+0000", 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0766e-05 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825460+0000", 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111922 2024-09-22T01:15:43.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.825494+0000", 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4377e-05 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840024+0000", 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014530134 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:43.840077+0000", 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2403e-05 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:121385 3.8 3:10090572:::rbd_data.11248450c2ab.000000000000008b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.284877+0000", 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 6.7011101870000003, 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014577705, 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 121385 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284877+0000", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284877+0000", 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284877+0000", 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284881+0000", 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.41e-06 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284891+0000", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8989999999999996e-06 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.284902+0000", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1314999999999999e-05 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.295991+0000", 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011088893000000001 2024-09-22T01:15:43.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.296289+0000", 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029814899999999999 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.299454+0000", 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.003165359 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:124421 3.1 3:82d045fd:::rbd_data.112449de4852.0000000000000005:head [delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:38.118713+0000", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.8672737999999995, 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017204420000000002, 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124421 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118713+0000", 2024-09-22T01:15:43.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118713+0000", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118716+0000", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118724+0000", 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6130000000000004e-06 2024-09-22T01:15:43.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118746+0000", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1484999999999998e-05 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.118761+0000", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5260999999999998e-05 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.122029+0000", 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0032677169999999998 2024-09-22T01:15:43.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.122258+0000", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022893800000000001 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.123105+0000", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00084683399999999998 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.129597+0000", 2024-09-22T01:15:43.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0064926719999999997 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.135667+0000", 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0060698970000000003 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.135808+0000", 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00014066 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.135918+0000", 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109623 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:124423 3.c e100/24 3:372ecc9e:::rbd_data.112449de4852.0000000000000001:head v 100'490, mlcod=100'483)", 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:38.120008+0000", 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.8659785180000004, 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014449953, 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.069 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120008+0000", 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120008+0000", 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120009+0000", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120018+0000", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5080000000000005e-06 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120054+0000", 2024-09-22T01:15:43.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6523000000000003e-05 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.120067+0000", 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.221e-05 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.122791+0000", 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027240530000000002 2024-09-22T01:15:43.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.122840+0000", 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9750000000000003e-05 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.134364+0000", 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.011523927999999999 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.134458+0000", 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3978e-05 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:124461 3.6 3:65e513a5:::rbd_data.112449de4852.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:38.123373+0000", 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.8626137700000003, 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017568596999999998, 2024-09-22T01:15:43.074 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124461 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.123373+0000", 2024-09-22T01:15:43.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.123373+0000", 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.123376+0000", 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.824e-06 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124283+0000", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00090746499999999999 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124317+0000", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3114999999999997e-05 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124338+0000", 2024-09-22T01:15:43.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1492e-05 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124498+0000", 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015979899999999999 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124710+0000", 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021185 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125229+0000", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051957100000000003 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.134557+0000", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093280619999999998 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.140761+0000", 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0062041850000000001 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.140871+0000", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000109113 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.140942+0000", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1120999999999999e-05 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:124462 3.6 3:65e513a5:::rbd_data.112449de4852.0000000000000008:head [delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:38.124397+0000", 2024-09-22T01:15:43.080 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.8615895609999997, 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016850609999999999, 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124462 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.081 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124397+0000", 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124397+0000", 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124399+0000", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124408+0000", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.14e-06 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124429+0000", 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1464999999999999e-05 2024-09-22T01:15:43.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124447+0000", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8399000000000001e-05 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.128978+0000", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0045307639999999996 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.129326+0000", 2024-09-22T01:15:43.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00034812100000000001 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.129817+0000", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049080300000000005 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.138063+0000", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0082456860000000003 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.141039+0000", 2024-09-22T01:15:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0029759859999999999 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.141160+0000", 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000120797 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.141248+0000", 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8360000000000001e-05 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.086 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:124463 3.8 3:17acad14:::rbd_data.112449de4852.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:38.124471+0000", 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 4.8615161330000003, 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015275867, 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 124463 2024-09-22T01:15:43.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124471+0000", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124471+0000", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.124472+0000", 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.207e-06 2024-09-22T01:15:43.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125161+0000", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000688914 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125190+0000", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8901e-05 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125209+0000", 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9126e-05 2024-09-22T01:15:43.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125380+0000", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00017110500000000001 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.125598+0000", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000217689 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.126118+0000", 2024-09-22T01:15:43.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00052021800000000003 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.134624+0000", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0085056860000000001 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.139478+0000", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0048546819999999999 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.139636+0000", 2024-09-22T01:15:43.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015813000000000001 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:38.139747+0000", 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000110209 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:43.092 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:43.093 INFO:teuthology.orchestra.run.smithi096.stdout:5666 punch from 0x255cfcb to 0x2566e8c, (0x9ec1 bytes) 2024-09-22T01:15:43.093 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:15:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:15:43.095 INFO:teuthology.orchestra.run.smithi096.stdout:5667 trunc from 0xc98c215 to 0x12dba9f 2024-09-22T01:15:43.095 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:43.089+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:43.095 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:43.096 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:43.097 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:15:43.098 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:43.098 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:43.098 INFO:teuthology.orchestra.run.smithi096.stdout:5668 writesame 0xdaa5d37 thru 0xdab30e7 data_size 0x1(0xd3b1 bytes) 2024-09-22T01:15:43.099 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.099 INFO:teuthology.orchestra.run.smithi096.stdout:5675 punch from 0x9c1195d to 0x9c1356f, (0x1c12 bytes) 2024-09-22T01:15:43.099 INFO:teuthology.orchestra.run.smithi096.stdout:5676 writesame 0x9937084 thru 0x993ab14 data_size 0xb(0x3a91 bytes) 2024-09-22T01:15:43.100 INFO:teuthology.orchestra.run.smithi096.stdout:5678 write 0xb1b2d52 thru 0xb1ba831 (0x7ae0 bytes) 2024-09-22T01:15:43.103 INFO:teuthology.orchestra.run.smithi096.stdout:5679 punch from 0x249ef56 to 0x24ae388, (0xf432 bytes) 2024-09-22T01:15:43.103 INFO:teuthology.orchestra.run.smithi096.stdout:5681 writesame 0x297e856 thru 0x2981600 data_size 0x9(0x2dab bytes) 2024-09-22T01:15:43.114 INFO:teuthology.orchestra.run.smithi096.stdout:5682 read 0xb0f011f thru 0xb0f45d0 (0x44b2 bytes) 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:5683 read 0x1600010 thru 0x1607f0f (0x7f00 bytes) 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:5687 read 0x168af6a thru 0x168c8b5 (0x194c bytes) 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:5689 read 0x7756b64 thru 0x7758753 (0x1bf0 bytes) 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:5690 punch from 0xd7d3e36 to 0xd7e0d2e, (0xcef8 bytes) 2024-09-22T01:15:43.115 INFO:teuthology.orchestra.run.smithi096.stdout:5691 writesame 0x8580104 thru 0x8584882 data_size 0x1(0x477f bytes) 2024-09-22T01:15:43.133 INFO:teuthology.orchestra.run.smithi096.stdout:5693 read 0x2f8db60 thru 0x2f90bff (0x30a0 bytes) 2024-09-22T01:15:43.134 INFO:teuthology.orchestra.run.smithi096.stdout:5695 write 0xd88417a thru 0xd887c52 (0x3ad9 bytes) 2024-09-22T01:15:43.163 INFO:teuthology.orchestra.run.smithi096.stdout:5696 trunc from 0xdab30e8 to 0x218987a 2024-09-22T01:15:43.194 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:43.188+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:43.203 INFO:teuthology.orchestra.run.smithi096.stdout:5697 read 0x1318ac3 thru 0x131d611 (0x4b4f bytes) 2024-09-22T01:15:43.219 INFO:teuthology.orchestra.run.smithi096.stdout:5699 trunc from 0x218987a to 0xc8c36d4 2024-09-22T01:15:43.233 INFO:teuthology.orchestra.run.smithi096.stdout:5701 write 0x46de37c thru 0x46e0f50 (0x2bd5 bytes) 2024-09-22T01:15:43.234 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:43.235 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:43.248 INFO:teuthology.orchestra.run.smithi096.stdout:5702 writesame 0x1939aae thru 0x1949614 data_size 0x3(0xfb67 bytes) 2024-09-22T01:15:43.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:43.268 INFO:teuthology.orchestra.run.smithi096.stdout:5704 punch from 0x3b82d7d to 0x3b90b2b, (0xddae bytes) 2024-09-22T01:15:43.268 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.269 INFO:teuthology.orchestra.run.smithi096.stdout:5707 trunc from 0xc8c36d4 to 0xba6554b 2024-09-22T01:15:43.285 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.285 INFO:teuthology.orchestra.run.smithi096.stdout:5709 punch from 0xb9295ec to 0xb92c170, (0x2b84 bytes) 2024-09-22T01:15:43.285 INFO:teuthology.orchestra.run.smithi096.stdout:5710 punch from 0x1976a3b to 0x197e5ef, (0x7bb4 bytes) 2024-09-22T01:15:43.285 INFO:teuthology.orchestra.run.smithi096.stdout:5711 read 0x8feb72 thru 0x901f24 (0x33b3 bytes) 2024-09-22T01:15:43.286 INFO:teuthology.orchestra.run.smithi096.stdout:5712 writesame 0xb67a715 thru 0xb67b007 data_size 0x1(0x8f3 bytes) 2024-09-22T01:15:43.296 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:43.293+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:43.296 INFO:teuthology.orchestra.run.smithi096.stdout:5717 write 0x8cb1393 thru 0x8cbdcd7 (0xc945 bytes) 2024-09-22T01:15:43.315 INFO:teuthology.orchestra.run.smithi096.stdout:5719 trunc from 0xba6554b to 0xa73422e 2024-09-22T01:15:43.339 INFO:teuthology.orchestra.run.smithi096.stdout:5720 trunc from 0xa73422e to 0xd85a8a3 2024-09-22T01:15:43.349 INFO:teuthology.orchestra.run.smithi096.stdout:5721 write 0xc54c601 thru 0xc54d4e1 (0xee1 bytes) 2024-09-22T01:15:43.356 INFO:teuthology.orchestra.run.smithi096.stdout:5722 punch from 0xa38d8a0 to 0xa38f5c6, (0x1d26 bytes) 2024-09-22T01:15:43.356 INFO:teuthology.orchestra.run.smithi096.stdout:5723 read 0xc8bacd thru 0xc8dce1 (0x2215 bytes) 2024-09-22T01:15:43.357 INFO:teuthology.orchestra.run.smithi096.stdout:5724 punch from 0x40432ea to 0x404e255, (0xaf6b bytes) 2024-09-22T01:15:43.358 INFO:teuthology.orchestra.run.smithi096.stdout:5725 write 0x25af0c9 thru 0x25b7917 (0x884f bytes) 2024-09-22T01:15:43.375 INFO:teuthology.orchestra.run.smithi096.stdout:5728 writesame 0x46189b thru 0x46b6c5 data_size 0x21(0x9e2b bytes) 2024-09-22T01:15:43.392 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:43.393 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:43.397 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:43.393+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:43.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:43.498 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:43.497+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:43.563 INFO:teuthology.orchestra.run.smithi096.stdout:5730 write 0x4729f2f thru 0x472fd57 (0x5e29 bytes) 2024-09-22T01:15:43.573 INFO:teuthology.orchestra.run.smithi096.stdout:5731 read 0x64f44a0 thru 0x64fd185 (0x8ce6 bytes) 2024-09-22T01:15:43.573 INFO:teuthology.orchestra.run.smithi096.stdout:5732 punch from 0x887491 to 0x892a1a, (0xb589 bytes) 2024-09-22T01:15:43.573 INFO:teuthology.orchestra.run.smithi096.stdout:5733 write 0xa7105df thru 0xa710dff (0x821 bytes) 2024-09-22T01:15:43.582 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:43.582 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:43.584 INFO:teuthology.orchestra.run.smithi096.stdout:5734 write 0xb802b57 thru 0xb810129 (0xd5d3 bytes) 2024-09-22T01:15:43.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:43.601 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:43.596+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:43.602 INFO:teuthology.orchestra.run.smithi096.stdout:5735 write 0x6865fe3 thru 0x686a399 (0x43b7 bytes) 2024-09-22T01:15:43.621 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.621 INFO:teuthology.orchestra.run.smithi096.stdout:5737 write 0xbe6e7b1 thru 0xbe705af (0x1dff bytes) 2024-09-22T01:15:43.636 INFO:teuthology.orchestra.run.smithi096.stdout:5741 writesame 0x94bb86e thru 0x94c2f8b data_size 0x2(0x771e bytes) 2024-09-22T01:15:43.649 INFO:teuthology.orchestra.run.smithi096.stdout:5742 write 0x129ec3 thru 0x12b29f (0x13dd bytes) 2024-09-22T01:15:43.700 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:43.696+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:43.745 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:43.746 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:43.761 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:43.773 INFO:teuthology.orchestra.run.smithi096.stdout:5743 writesame 0x8175ba4 thru 0x8178023 data_size 0x8(0x2480 bytes) 2024-09-22T01:15:43.785 INFO:teuthology.orchestra.run.smithi096.stdout:5744 read 0x15c9c64 thru 0x15d46fc (0xaa99 bytes) 2024-09-22T01:15:43.787 INFO:teuthology.orchestra.run.smithi096.stdout:5745 trunc from 0xd85a8a3 to 0xc550de3 2024-09-22T01:15:43.801 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:43.796+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:43.807 INFO:teuthology.orchestra.run.smithi096.stdout:5748 trunc from 0xc550de3 to 0x73a9d60 2024-09-22T01:15:43.852 INFO:teuthology.orchestra.run.smithi096.stdout:5749 trunc from 0x73a9d60 to 0xd58af4b 2024-09-22T01:15:43.864 INFO:teuthology.orchestra.run.smithi096.stdout:5751 punch from 0x2f9183d to 0x2f9ca69, (0xb22c bytes) 2024-09-22T01:15:43.864 INFO:teuthology.orchestra.run.smithi096.stdout:5752 read 0x7f72714 thru 0x7f72de8 (0x6d5 bytes) 2024-09-22T01:15:43.865 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:43.865 INFO:teuthology.orchestra.run.smithi096.stdout:5754 read 0x84dd2d5 thru 0x84e8df4 (0xbb20 bytes) 2024-09-22T01:15:43.865 INFO:teuthology.orchestra.run.smithi096.stdout:5756 writesame 0x28fda64 thru 0x290d164 data_size 0x1(0xf701 bytes) 2024-09-22T01:15:43.900 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:43.900 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:43.902 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:43.896+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:43.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:43.920 INFO:teuthology.orchestra.run.smithi096.stdout:5757 trunc from 0xd58af4b to 0x92b040d 2024-09-22T01:15:43.934 INFO:teuthology.orchestra.run.smithi096.stdout:5758 clone 49 order 25 su 16777216 sc 5 2024-09-22T01:15:44.003 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:43.996+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:44.022 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:15:44.048 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:44.048 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:44.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:15:44.063 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:44.104 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:44.101+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:44.139 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone48 from 0x92b040d (overlap 0x12dba9f) to 0x0 2024-09-22T01:15:44.199 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:44.199 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:44.205 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:44.201+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:44.218 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:44.305 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:44.300+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:44.406 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:44.400+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:44.407 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:44.408 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:44.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:44.508 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:44.504+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:44.609 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:44.605+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:44.635 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:44.635 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:44.655 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:44.740 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:44.736+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:44.810 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:44.804+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:44.846 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:44.846 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:44.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:44.912 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:44.909+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:45.012 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:45.008+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:45.030 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:15:45.043 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:45.043 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:45.058 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:15:45.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:15:45.063 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:45.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:15:45.100 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:45.113 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:45.109+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:45.213 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:45.212+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:45.290 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:45.290 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:45.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:45.356 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:45.356+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:45.362 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-22T01:15:45.365 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:45.395 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:45.396 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:45.414 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:45.413+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:45.427 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:45.427 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:45.445 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:45.446 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:45.459 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:45.459 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:45.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:45.493 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:45.493 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:45.512 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:45.512 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:45.530 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:45.530 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:45.546 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:45.545+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:45.553 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:45.553 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:45.563 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:15:45.580 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-22T01:15:45.028844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468699132,"num_objects":1937,"num_object_clones":513,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":649,"num_read":179733,"num_read_kb":848426,"num_write":17844,"num_write_kb":342113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":46033975,"num_keys_recovered":155,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14472,"ondisk_log_size":14472,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":827556,"kb_used_data":480812,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754147164,"statfs":{"total":773094113280,"available":772246695936,"internally_reserved":0,"allocated":492351488,"data_stored":666110638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":354799152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65722660,"num_objects":115,"num_object_clones":18,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":26,"num_read":8043,"num_read_kb":28144,"num_write":973,"num_write_kb":11974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.037942"},"pg_stats":[{"pgid":"3.1f","version":"105'214","reported_seq":1670,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.244634+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:15:43.244634+0000","last_peered":"2024-09-22T01:15:43.244634+0000","last_clean":"2024-09-22T01:15:43.244634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:15:43.244634+0000","last_undegraded":"2024-09-22T01:15:43.244634+0000","last_fullsized":"2024-09-22T01:15:43.244634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:38.232159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":19099330,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":1350,"num_read_kb":19823,"num_write":215,"num_write_kb":5062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"105'376","reported_seq":11563,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.124307+0000","last_change":"2024-09-22T01:14:38.655424+0000","last_active":"2024-09-22T01:15:43.124307+0000","last_peered":"2024-09-22T01:15:43.124307+0000","last_clean":"2024-09-22T01:15:43.124307+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:15:43.124307+0000","last_undegraded":"2024-09-22T01:15:43.124307+0000","last_fullsized":"2024-09-22T01:15:43.124307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:52:30.570084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":52883936,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":11401,"num_read_kb":35290,"num_write":441,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"105'273","reported_seq":12732,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204539+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:15:43.204539+0000","last_peered":"2024-09-22T01:15:43.204539+0000","last_clean":"2024-09-22T01:15:43.204539+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:15:43.204539+0000","last_undegraded":"2024-09-22T01:15:43.204539+0000","last_fullsized":"2024-09-22T01:15:43.204539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:06.730037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":31911709,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":13501,"num_read_kb":24099,"num_write":292,"num_write_kb":9584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"105'472","reported_seq":4416,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.557766+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:15:43.557766+0000","last_peered":"2024-09-22T01:15:43.557766+0000","last_clean":"2024-09-22T01:15:43.557766+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:15:43.557766+0000","last_undegraded":"2024-09-22T01:15:43.557766+0000","last_fullsized":"2024-09-22T01:15:43.557766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:55:53.401122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":48416139,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":5105,"num_read_kb":22376,"num_write":582,"num_write_kb":12492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"104'241","reported_seq":2704,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.013906+0000","last_change":"2024-09-22T01:15:32.944791+0000","last_active":"2024-09-22T01:15:42.013906+0000","last_peered":"2024-09-22T01:15:42.013906+0000","last_clean":"2024-09-22T01:15:42.013906+0000","last_became_active":"2024-09-22T01:15:32.940539+0000","last_became_peered":"2024-09-22T01:15:32.940539+0000","last_unstale":"2024-09-22T01:15:42.013906+0000","last_undegraded":"2024-09-22T01:15:42.013906+0000","last_fullsized":"2024-09-22T01:15:42.013906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:23.280647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":50907626,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":2632,"num_read_kb":7585,"num_write":212,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"105'948","reported_seq":14288,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.335192+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:15:43.335192+0000","last_peered":"2024-09-22T01:15:43.335192+0000","last_clean":"2024-09-22T01:15:43.335192+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:15:43.335192+0000","last_undegraded":"2024-09-22T01:15:43.335192+0000","last_fullsized":"2024-09-22T01:15:43.335192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:46:15.000880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":68265803,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":18,"num_read":17349,"num_read_kb":74988,"num_write":1557,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"105'294","reported_seq":4276,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.519457+0000","last_change":"2024-09-22T01:15:32.938982+0000","last_active":"2024-09-22T01:15:43.519457+0000","last_peered":"2024-09-22T01:15:43.519457+0000","last_clean":"2024-09-22T01:15:43.519457+0000","last_became_active":"2024-09-22T01:15:32.936966+0000","last_became_peered":"2024-09-22T01:15:32.936966+0000","last_unstale":"2024-09-22T01:15:43.519457+0000","last_undegraded":"2024-09-22T01:15:43.519457+0000","last_fullsized":"2024-09-22T01:15:43.519457+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:23:29.158081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":63704915,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":5299,"num_read_kb":25751,"num_write":270,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"105'243","reported_seq":4711,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.296955+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:15:43.296955+0000","last_peered":"2024-09-22T01:15:43.296955+0000","last_clean":"2024-09-22T01:15:43.296955+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:15:43.296955+0000","last_undegraded":"2024-09-22T01:15:43.296955+0000","last_fullsized":"2024-09-22T01:15:43.296955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:23.653601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":52002410,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4585,"num_read_kb":37483,"num_write":243,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"105'350","reported_seq":3593,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.348406+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:15:43.348406+0000","last_peered":"2024-09-22T01:15:43.348406+0000","last_clean":"2024-09-22T01:15:43.348406+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:15:43.348406+0000","last_undegraded":"2024-09-22T01:15:43.348406+0000","last_fullsized":"2024-09-22T01:15:43.348406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:58:18.121857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":67247509,"num_objects":61,"num_object_clones":19,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":4133,"num_read_kb":32002,"num_write":374,"num_write_kb":11548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"105'348","reported_seq":10929,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.312299+0000","last_change":"2024-09-22T01:14:41.577883+0000","last_active":"2024-09-22T01:15:43.312299+0000","last_peered":"2024-09-22T01:15:43.312299+0000","last_clean":"2024-09-22T01:15:43.312299+0000","last_became_active":"2024-09-22T01:14:41.576276+0000","last_became_peered":"2024-09-22T01:14:41.576276+0000","last_unstale":"2024-09-22T01:15:43.312299+0000","last_undegraded":"2024-09-22T01:15:43.312299+0000","last_fullsized":"2024-09-22T01:15:43.312299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:31:07.544940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":47229034,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":23,"num_read":10610,"num_read_kb":24970,"num_write":398,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4954128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"104'398","reported_seq":1914,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.554512+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:15:42.554512+0000","last_peered":"2024-09-22T01:15:42.554512+0000","last_clean":"2024-09-22T01:15:42.554512+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:15:42.554512+0000","last_undegraded":"2024-09-22T01:15:42.554512+0000","last_fullsized":"2024-09-22T01:15:42.554512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:52.641561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":21637909,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":2685,"num_read_kb":15219,"num_write":548,"num_write_kb":3347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"105'583","reported_seq":2244,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.358211+0000","last_change":"2024-09-22T01:14:41.579663+0000","last_active":"2024-09-22T01:15:43.358211+0000","last_peered":"2024-09-22T01:15:43.358211+0000","last_clean":"2024-09-22T01:15:43.358211+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:15:43.358211+0000","last_undegraded":"2024-09-22T01:15:43.358211+0000","last_fullsized":"2024-09-22T01:15:43.358211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:56:45.547334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":65348068,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":4138,"num_read_kb":19680,"num_write":802,"num_write_kb":11534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014337+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:15:43.014337+0000","last_peered":"2024-09-22T01:15:43.014337+0000","last_clean":"2024-09-22T01:15:43.014337+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:15:43.014337+0000","last_undegraded":"2024-09-22T01:15:43.014337+0000","last_fullsized":"2024-09-22T01:15:43.014337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"105'398","reported_seq":2940,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014118+0000","last_change":"2024-09-22T01:15:32.938608+0000","last_active":"2024-09-22T01:15:43.014118+0000","last_peered":"2024-09-22T01:15:43.014118+0000","last_clean":"2024-09-22T01:15:43.014118+0000","last_became_active":"2024-09-22T01:15:32.937080+0000","last_became_peered":"2024-09-22T01:15:32.937080+0000","last_unstale":"2024-09-22T01:15:43.014118+0000","last_undegraded":"2024-09-22T01:15:43.014118+0000","last_fullsized":"2024-09-22T01:15:43.014118+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:12.539266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":35080637,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":5150,"num_read_kb":39840,"num_write":377,"num_write_kb":22695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.014055+0000","last_change":"2024-09-22T01:15:32.938118+0000","last_active":"2024-09-22T01:15:42.014055+0000","last_peered":"2024-09-22T01:15:42.014055+0000","last_clean":"2024-09-22T01:15:42.014055+0000","last_became_active":"2024-09-22T01:15:32.937355+0000","last_became_peered":"2024-09-22T01:15:32.937355+0000","last_unstale":"2024-09-22T01:15:42.014055+0000","last_undegraded":"2024-09-22T01:15:42.014055+0000","last_fullsized":"2024-09-22T01:15:42.014055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:32.494366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"105'228","reported_seq":2127,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.371867+0000","last_change":"2024-09-22T01:15:32.928350+0000","last_active":"2024-09-22T01:15:43.371867+0000","last_peered":"2024-09-22T01:15:43.371867+0000","last_clean":"2024-09-22T01:15:43.371867+0000","last_became_active":"2024-09-22T01:15:32.926099+0000","last_became_peered":"2024-09-22T01:15:32.926099+0000","last_unstale":"2024-09-22T01:15:43.371867+0000","last_undegraded":"2024-09-22T01:15:43.371867+0000","last_fullsized":"2024-09-22T01:15:43.371867+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:07:04.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":34095983,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":21,"num_read":2337,"num_read_kb":24321,"num_write":218,"num_write_kb":9788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014520+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:15:43.014520+0000","last_peered":"2024-09-22T01:15:43.014520+0000","last_clean":"2024-09-22T01:15:43.014520+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:15:43.014520+0000","last_undegraded":"2024-09-22T01:15:43.014520+0000","last_fullsized":"2024-09-22T01:15:43.014520+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"105'479","reported_seq":2324,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.030485+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:15:43.030485+0000","last_peered":"2024-09-22T01:15:43.030485+0000","last_clean":"2024-09-22T01:15:43.030485+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:15:43.030485+0000","last_undegraded":"2024-09-22T01:15:43.030485+0000","last_fullsized":"2024-09-22T01:15:43.030485+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:23.860202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":29917963,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":25,"num_read":4402,"num_read_kb":18549,"num_write":664,"num_write_kb":6139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":181,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.026884+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:15:43.026884+0000","last_peered":"2024-09-22T01:15:43.026884+0000","last_clean":"2024-09-22T01:15:43.026884+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:15:43.026884+0000","last_undegraded":"2024-09-22T01:15:43.026884+0000","last_fullsized":"2024-09-22T01:15:43.026884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"105'466","reported_seq":3218,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.584732+0000","last_change":"2024-09-22T01:14:41.570935+0000","last_active":"2024-09-22T01:15:43.584732+0000","last_peered":"2024-09-22T01:15:43.584732+0000","last_clean":"2024-09-22T01:15:43.584732+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:15:43.584732+0000","last_undegraded":"2024-09-22T01:15:43.584732+0000","last_fullsized":"2024-09-22T01:15:43.584732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:24.829918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":43247794,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":4075,"num_read_kb":29894,"num_write":612,"num_write_kb":10687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"104'504","reported_seq":3130,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.243185+0000","last_change":"2024-09-22T01:14:38.704568+0000","last_active":"2024-09-22T01:15:42.243185+0000","last_peered":"2024-09-22T01:15:42.243185+0000","last_clean":"2024-09-22T01:15:42.243185+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:15:42.243185+0000","last_undegraded":"2024-09-22T01:15:42.243185+0000","last_fullsized":"2024-09-22T01:15:42.243185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:39.006012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":35415539,"num_objects":75,"num_object_clones":22,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":23,"num_read":3867,"num_read_kb":25426,"num_write":636,"num_write_kb":10113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":166,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014475+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:15:43.014475+0000","last_peered":"2024-09-22T01:15:43.014475+0000","last_clean":"2024-09-22T01:15:43.014475+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:15:43.014475+0000","last_undegraded":"2024-09-22T01:15:43.014475+0000","last_fullsized":"2024-09-22T01:15:43.014475+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"105'341","reported_seq":2016,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.176886+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:15:43.176886+0000","last_peered":"2024-09-22T01:15:43.176886+0000","last_clean":"2024-09-22T01:15:43.176886+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:15:43.176886+0000","last_undegraded":"2024-09-22T01:15:43.176886+0000","last_fullsized":"2024-09-22T01:15:43.176886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:38:25.066856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":38351373,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":2657,"num_read_kb":25711,"num_write":386,"num_write_kb":7606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014123+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:15:43.014123+0000","last_peered":"2024-09-22T01:15:43.014123+0000","last_clean":"2024-09-22T01:15:43.014123+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:15:43.014123+0000","last_undegraded":"2024-09-22T01:15:43.014123+0000","last_fullsized":"2024-09-22T01:15:43.014123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"105'347","reported_seq":1859,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204699+0000","last_change":"2024-09-22T01:15:32.940653+0000","last_active":"2024-09-22T01:15:43.204699+0000","last_peered":"2024-09-22T01:15:43.204699+0000","last_clean":"2024-09-22T01:15:43.204699+0000","last_became_active":"2024-09-22T01:15:32.937217+0000","last_became_peered":"2024-09-22T01:15:32.937217+0000","last_unstale":"2024-09-22T01:15:43.204699+0000","last_undegraded":"2024-09-22T01:15:43.204699+0000","last_fullsized":"2024-09-22T01:15:43.204699+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:18.263018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":28367676,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":3012,"num_read_kb":7898,"num_write":372,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014071+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:15:43.014071+0000","last_peered":"2024-09-22T01:15:43.014071+0000","last_clean":"2024-09-22T01:15:43.014071+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:15:43.014071+0000","last_undegraded":"2024-09-22T01:15:43.014071+0000","last_fullsized":"2024-09-22T01:15:43.014071+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"105'439","reported_seq":5261,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.031926+0000","last_change":"2024-09-22T01:15:32.932715+0000","last_active":"2024-09-22T01:15:43.031926+0000","last_peered":"2024-09-22T01:15:43.031926+0000","last_clean":"2024-09-22T01:15:43.031926+0000","last_became_active":"2024-09-22T01:15:32.929890+0000","last_became_peered":"2024-09-22T01:15:32.929890+0000","last_unstale":"2024-09-22T01:15:43.031926+0000","last_undegraded":"2024-09-22T01:15:43.031926+0000","last_fullsized":"2024-09-22T01:15:43.031926+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:17:34.222484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":85750894,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":19,"num_read":6922,"num_read_kb":24445,"num_write":511,"num_write_kb":9638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":184,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.016319+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:15:43.016319+0000","last_peered":"2024-09-22T01:15:43.016319+0000","last_clean":"2024-09-22T01:15:43.016319+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:15:43.016319+0000","last_undegraded":"2024-09-22T01:15:43.016319+0000","last_fullsized":"2024-09-22T01:15:43.016319+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":333,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.024730+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:15:43.024730+0000","last_peered":"2024-09-22T01:15:43.024730+0000","last_clean":"2024-09-22T01:15:43.024730+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:15:43.024730+0000","last_undegraded":"2024-09-22T01:15:43.024730+0000","last_fullsized":"2024-09-22T01:15:43.024730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"105'322","reported_seq":1682,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.307396+0000","last_change":"2024-09-22T01:15:32.939125+0000","last_active":"2024-09-22T01:15:43.307396+0000","last_peered":"2024-09-22T01:15:43.307396+0000","last_clean":"2024-09-22T01:15:43.307396+0000","last_became_active":"2024-09-22T01:15:32.937085+0000","last_became_peered":"2024-09-22T01:15:32.937085+0000","last_unstale":"2024-09-22T01:15:43.307396+0000","last_undegraded":"2024-09-22T01:15:43.307396+0000","last_fullsized":"2024-09-22T01:15:43.307396+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:54.725390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":63093377,"num_objects":66,"num_object_clones":23,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":22,"num_read":1962,"num_read_kb":14598,"num_write":266,"num_write_kb":3876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"105'750","reported_seq":16569,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.133806+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:15:43.133806+0000","last_peered":"2024-09-22T01:15:43.133806+0000","last_clean":"2024-09-22T01:15:43.133806+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:15:43.133806+0000","last_undegraded":"2024-09-22T01:15:43.133806+0000","last_fullsized":"2024-09-22T01:15:43.133806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:44.800290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":93056327,"num_objects":81,"num_object_clones":21,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":21212,"num_read_kb":78250,"num_write":802,"num_write_kb":22521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"105'395","reported_seq":2577,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014443+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:15:43.014443+0000","last_peered":"2024-09-22T01:15:43.014443+0000","last_clean":"2024-09-22T01:15:43.014443+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:15:43.014443+0000","last_undegraded":"2024-09-22T01:15:43.014443+0000","last_fullsized":"2024-09-22T01:15:43.014443+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:50:35.904065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":27862245,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":5186,"num_read_kb":9876,"num_write":387,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"105'404","reported_seq":3005,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.187255+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:15:43.187255+0000","last_peered":"2024-09-22T01:15:43.187255+0000","last_clean":"2024-09-22T01:15:43.187255+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:15:43.187255+0000","last_undegraded":"2024-09-22T01:15:43.187255+0000","last_fullsized":"2024-09-22T01:15:43.187255+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:55.321982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":42504998,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":4098,"num_read_kb":16291,"num_write":548,"num_write_kb":8521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"104'509","reported_seq":3015,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.013977+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:15:42.013977+0000","last_peered":"2024-09-22T01:15:42.013977+0000","last_clean":"2024-09-22T01:15:42.013977+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:15:42.013977+0000","last_undegraded":"2024-09-22T01:15:42.013977+0000","last_fullsized":"2024-09-22T01:15:42.013977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:54.089887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":35868587,"num_objects":84,"num_object_clones":29,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":34,"num_read":3503,"num_read_kb":26065,"num_write":634,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"105'542","reported_seq":1469,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.262783+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:15:43.262783+0000","last_peered":"2024-09-22T01:15:43.262783+0000","last_clean":"2024-09-22T01:15:43.262783+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:15:43.262783+0000","last_undegraded":"2024-09-22T01:15:43.262783+0000","last_fullsized":"2024-09-22T01:15:43.262783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:28.428817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":22060365,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":2867,"num_read_kb":11363,"num_write":769,"num_write_kb":6282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"105'696","reported_seq":2773,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.185119+0000","last_change":"2024-09-22T01:14:41.579603+0000","last_active":"2024-09-22T01:15:43.185119+0000","last_peered":"2024-09-22T01:15:43.185119+0000","last_clean":"2024-09-22T01:15:43.185119+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:15:43.185119+0000","last_undegraded":"2024-09-22T01:15:43.185119+0000","last_fullsized":"2024-09-22T01:15:43.185119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:37.649550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":34397159,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":22,"num_read":4429,"num_read_kb":20364,"num_write":962,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"104'477","reported_seq":1241,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.014032+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:15:42.014032+0000","last_peered":"2024-09-22T01:15:42.014032+0000","last_clean":"2024-09-22T01:15:42.014032+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:15:42.014032+0000","last_undegraded":"2024-09-22T01:15:42.014032+0000","last_fullsized":"2024-09-22T01:15:42.014032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:41:44.236986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":32194018,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":1988,"num_read_kb":19901,"num_write":643,"num_write_kb":10263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"105'454","reported_seq":2289,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204447+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:15:43.204447+0000","last_peered":"2024-09-22T01:15:43.204447+0000","last_clean":"2024-09-22T01:15:43.204447+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:15:43.204447+0000","last_undegraded":"2024-09-22T01:15:43.204447+0000","last_fullsized":"2024-09-22T01:15:43.204447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:04:45.370302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":34812101,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":20,"num_read":3569,"num_read_kb":14016,"num_write":555,"num_write_kb":4152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"105'726","reported_seq":3558,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.565672+0000","last_change":"2024-09-22T01:14:41.582658+0000","last_active":"2024-09-22T01:15:43.565672+0000","last_peered":"2024-09-22T01:15:43.565672+0000","last_clean":"2024-09-22T01:15:43.565672+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:15:43.565672+0000","last_undegraded":"2024-09-22T01:15:43.565672+0000","last_fullsized":"2024-09-22T01:15:43.565672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:40.979721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":47346680,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":13,"num_read":6271,"num_read_kb":48929,"num_write":979,"num_write_kb":22025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"105'618","reported_seq":3086,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.244811+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:15:43.244811+0000","last_peered":"2024-09-22T01:15:43.244811+0000","last_clean":"2024-09-22T01:15:43.244811+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:15:43.244811+0000","last_undegraded":"2024-09-22T01:15:43.244811+0000","last_fullsized":"2024-09-22T01:15:43.244811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:10:06.760100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":57271130,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":4829,"num_read_kb":24310,"num_write":814,"num_write_kb":10292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"105'517","reported_seq":2711,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.369541+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:15:43.369541+0000","last_peered":"2024-09-22T01:15:43.369541+0000","last_clean":"2024-09-22T01:15:43.369541+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:15:43.369541+0000","last_undegraded":"2024-09-22T01:15:43.369541+0000","last_fullsized":"2024-09-22T01:15:43.369541+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:36:45.318920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":58759511,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":26,"num_read":4493,"num_read_kb":29013,"num_write":614,"num_write_kb":5882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1468108745,"num_objects":1933,"num_object_clones":513,"num_object_copies":3866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1933,"num_whiteouts":649,"num_read":179617,"num_read_kb":848326,"num_write":17683,"num_write_kb":339695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":46033956,"num_keys_recovered":155,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":481640448,"data_stored":660597812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14352,"ondisk_log_size":14352,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79144,"kb_used_data":43792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94292696,"statfs":{"total":96636764160,"available":96555720704,"internally_reserved":0,"allocated":44843008,"data_stored":55912598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83344,"kb_used_data":43840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94288496,"statfs":{"total":96636764160,"available":96551419904,"internally_reserved":0,"allocated":44892160,"data_stored":55467226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111516,"kb_used_data":67840,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94260324,"statfs":{"total":96636764160,"available":96522571776,"internally_reserved":0,"allocated":69468160,"data_stored":91506573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107000,"kb_used_data":67488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94264840,"statfs":{"total":96636764160,"available":96527196160,"internally_reserved":0,"allocated":69107712,"data_stored":97500959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":73756,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":75526144,"data_stored":106058192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119608,"kb_used_data":70324,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94252232,"statfs":{"total":96636764160,"available":96514285568,"internally_reserved":0,"allocated":72011776,"data_stored":97976351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105368,"kb_used_data":60296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94266472,"statfs":{"total":96636764160,"available":96528867328,"internally_reserved":0,"allocated":61743104,"data_stored":86210706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98548,"kb_used_data":53476,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94273292,"statfs":{"total":96636764160,"available":96535851008,"internally_reserved":0,"allocated":54759424,"data_stored":75478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53567488,"data_stored":74925375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60026880,"data_stored":85129593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70836224,"data_stored":97435662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":74366976,"data_stored":105537461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67891200,"data_stored":96944923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":67637248,"data_stored":90348280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43712512,"data_stored":54940122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43601920,"data_stored":55336396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:15:45.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:15:45.617 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:45.613+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:45.621 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:45.622 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:45.622 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:45.622 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:45.622 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:45.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:45.717 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:45.716+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:45.740 INFO:teuthology.orchestra.run.smithi096.stdout:5759 writesame 0xbb5d1ad thru 0xbb67ca3 data_size 0x3(0xaaf7 bytes) 2024-09-22T01:15:45.741 INFO:teuthology.orchestra.run.smithi096.stdout:5760 clone 50 order 19 su 2048 sc 11 2024-09-22T01:15:45.811 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:45.811 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:45.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20619 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3014656~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455662164000003, 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052544992999999998, 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.812 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20619 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352571+0000", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6000000000000003e-07 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352575+0000", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5760000000000002e-06 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352584+0000", 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5089999999999999e-06 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352591+0000", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.109e-06 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404496+0000", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.051904761000000001 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.404604+0000", 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107587 2024-09-22T01:15:45.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405116+0000", 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00051179100000000005 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20620 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 3342336~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455631894000007, 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052904517999999998, 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.816 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20620 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352601+0000", 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5000000000000002e-07 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352605+0000", 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2200000000000001e-06 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352614+0000", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3950000000000007e-06 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352621+0000", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0929999999999997e-06 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405138+0000", 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052516902999999997 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405202+0000", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4345999999999997e-05 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405505+0000", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000302911 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20621 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 3670016~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:45.820 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455602321000001, 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053394728000000002, 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20621 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352630+0000", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352631+0000", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3e-07 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352634+0000", 2024-09-22T01:15:45.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9869999999999999e-06 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352653+0000", 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9358e-05 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352667+0000", 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3195e-05 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405546+0000", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052879255999999999 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.405607+0000", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1229999999999995e-05 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406025+0000", 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041807200000000002 2024-09-22T01:15:45.824 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20622 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4030464~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455548230000005, 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053841805999999999, 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.825 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20622 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352684+0000", 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352686+0000", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352691+0000", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7490000000000004e-06 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352709+0000", 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8151e-05 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352722+0000", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2249000000000001e-05 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406105+0000", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053383009000000002 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406170+0000", 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5677000000000002e-05 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406526+0000", 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000355874 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20623 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4358144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455492386000003, 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054352299999999999, 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.829 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20623 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352740+0000", 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352741+0000", 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352748+0000", 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4529999999999999e-06 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352766+0000", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8241e-05 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352780+0000", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3915e-05 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406589+0000", 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053808778000000002 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.406650+0000", 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0903000000000003e-05 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407093+0000", 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00044300699999999998 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20624 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 4685824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.833 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455434495999995, 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054850263000000003, 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20624 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352798+0000", 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352800+0000", 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4330000000000001e-06 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352806+0000", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0900000000000001e-06 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352828+0000", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2004000000000001e-05 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352842+0000", 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4457999999999999e-05 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407188+0000", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054345903000000001 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407326+0000", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00013819400000000001 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407648+0000", 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000322181 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.837 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20625 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5046272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455372763, 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055396642000000003, 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.838 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20625 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352860+0000", 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352861+0000", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0330000000000001e-06 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352878+0000", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352905+0000", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7780000000000002e-05 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352923+0000", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7963999999999999e-05 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407788+0000", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.054865091999999997 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.407882+0000", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.3567999999999996e-05 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408257+0000", 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00037463600000000002 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20626 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5373952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455277300000006, 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055953507, 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.842 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20626 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352955+0000", 2024-09-22T01:15:45.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352957+0000", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1999999999999999e-06 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352963+0000", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5359999999999998e-06 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352982+0000", 2024-09-22T01:15:45.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9117999999999999e-05 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.352997+0000", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5170999999999999e-05 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408327+0000", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055329991000000002 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408447+0000", 2024-09-22T01:15:45.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119917 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408909+0000", 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00046157400000000002 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20627 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 5701632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455215476999996, 2024-09-22T01:15:45.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056181144000000002, 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20627 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353017+0000", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353018+0000", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.003e-06 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353026+0000", 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9829999999999996e-06 2024-09-22T01:15:45.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353045+0000", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8448e-05 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353058+0000", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2971e-05 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.408973+0000", 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.055915486 2024-09-22T01:15:45.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409087+0000", 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000113785 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409198+0000", 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000111468 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.850 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20628 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [sparse-read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455124337000001, 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056267956000000001, 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20628 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353108+0000", 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353110+0000", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.443e-06 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353116+0000", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0229999999999998e-06 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353135+0000", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9553999999999999e-05 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.853 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353150+0000", 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4402e-05 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409242+0000", 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056091944999999997 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409300+0000", 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7964999999999998e-05 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409376+0000", 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6624000000000004e-05 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20629 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6389760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455062431000002, 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056487653999999998, 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.855 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20629 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353170+0000", 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353171+0000", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.074e-06 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353179+0000", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4449999999999999e-06 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353197+0000", 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7975e-05 2024-09-22T01:15:45.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353213+0000", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6055000000000001e-05 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409428+0000", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056214933000000002 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409531+0000", 2024-09-22T01:15:45.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103071 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409658+0000", 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012710099999999999 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20630 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 6717440~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.455001944000003, 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056581724, 2024-09-22T01:15:45.859 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20630 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353231+0000", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353232+0000", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.17e-06 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353237+0000", 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3650000000000003e-06 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353257+0000", 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0061999999999999e-05 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353270+0000", 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3025e-05 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409686+0000", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056415137999999997 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409738+0000", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2472999999999997e-05 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409812+0000", 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4491000000000002e-05 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.863 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20631 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7045120~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.454944474000001, 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056783186999999999, 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.864 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20631 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353288+0000", 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353289+0000", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0529999999999999e-06 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353295+0000", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5600000000000001e-06 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353312+0000", 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7218e-05 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353325+0000", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3217999999999999e-05 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409864+0000", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056538465000000003 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.409970+0000", 2024-09-22T01:15:45.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000106211 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410071+0000", 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000101462 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20632 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.454887193999994, 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056895124999999998, 2024-09-22T01:15:45.868 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20632 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353346+0000", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6700000000000002e-07 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353352+0000", 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5929999999999996e-06 2024-09-22T01:15:45.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353370+0000", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7428e-05 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353386+0000", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6435000000000001e-05 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410118+0000", 2024-09-22T01:15:45.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056732362000000001 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410183+0000", 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4366e-05 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410241+0000", 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7973999999999998e-05 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20633 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.45482767, 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056990100000000002, 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20633 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353405+0000", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353406+0000", 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.023e-06 2024-09-22T01:15:45.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353412+0000", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7529999999999998e-06 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353431+0000", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9664e-05 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353444+0000", 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2167999999999999e-05 2024-09-22T01:15:45.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410277+0000", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056833541000000001 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410330+0000", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.287e-05 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410395+0000", 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5080999999999996e-05 2024-09-22T01:15:45.876 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20634 3.6 3:63271155:::rbd_data.11249227f48c.0000000000000000:e [read 8060928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.454774786000002, 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057190733000000001, 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.877 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20634 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353458+0000", 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353459+0000", 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7999999999999995e-07 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353462+0000", 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6689999999999999e-06 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353473+0000", 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0256e-05 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.879 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.353481+0000", 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5050000000000007e-06 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410455+0000", 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.056973585 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.880 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410552+0000", 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7017999999999998e-05 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410649+0000", 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.7020000000000003e-05 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.881 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:20780 3.6 3:63c17d97:::rbd_data.11247e2569dd.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.445662224000003, 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.057127731000000001, 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 20780 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.882 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362570+0000", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.362573+0000", 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3080000000000002e-06 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.883 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367753+0000", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0051802100000000002 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367780+0000", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7248999999999999e-05 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.367799+0000", 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655e-05 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.884 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410696+0000", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.042897289999999998 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.410836+0000", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000140276 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 7", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.411110+0000", 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000273646 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.885 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419282+0000", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0081716559999999994 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419576+0000", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00029393300000000002 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419643+0000", 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7310000000000004e-05 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.886 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:29.419698+0000", 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5198000000000002e-05 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.002403037999997, 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.887 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.888 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.889 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:15:45.891 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 40.981331257999997, 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053720695999999998, 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "waiting for ondisk", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:45.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826901+0000", 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826903+0000", 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.813e-06 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826916+0000", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2737e-05 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826938+0000", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2322000000000001e-05 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.826953+0000", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4379000000000001e-05 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.894 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827078+0000", 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012561199999999999 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for ondisk", 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.827187+0000", 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010877199999999999 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880622+0000", 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.053435060999999999 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.895 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 32.795013699000002, 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.896 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:15:45.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:15:45.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:15:45.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:45.900 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:45.902 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:45.903 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:15:45.903 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:45.904 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:45.900+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:45.914 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-22T01:15:45.557930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1468699132,"num_objects":1937,"num_object_clones":513,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":649,"num_read":179733,"num_read_kb":848426,"num_write":17844,"num_write_kb":342113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":46033975,"num_keys_recovered":155,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14472,"ondisk_log_size":14472,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":827556,"kb_used_data":480812,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754147164,"statfs":{"total":773094113280,"available":772246695936,"internally_reserved":0,"allocated":492351488,"data_stored":666110638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":354799152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65722660,"num_objects":115,"num_object_clones":18,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":26,"num_read":8043,"num_read_kb":28144,"num_write":973,"num_write_kb":11974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007091"},"pg_stats":[{"pgid":"3.1f","version":"105'214","reported_seq":1670,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.244634+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:15:43.244634+0000","last_peered":"2024-09-22T01:15:43.244634+0000","last_clean":"2024-09-22T01:15:43.244634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:15:43.244634+0000","last_undegraded":"2024-09-22T01:15:43.244634+0000","last_fullsized":"2024-09-22T01:15:43.244634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:38.232159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":19099330,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":1350,"num_read_kb":19823,"num_write":215,"num_write_kb":5062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"105'376","reported_seq":11563,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.124307+0000","last_change":"2024-09-22T01:14:38.655424+0000","last_active":"2024-09-22T01:15:43.124307+0000","last_peered":"2024-09-22T01:15:43.124307+0000","last_clean":"2024-09-22T01:15:43.124307+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:15:43.124307+0000","last_undegraded":"2024-09-22T01:15:43.124307+0000","last_fullsized":"2024-09-22T01:15:43.124307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:52:30.570084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":52883936,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":11401,"num_read_kb":35290,"num_write":441,"num_write_kb":15552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"105'273","reported_seq":12732,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204539+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:15:43.204539+0000","last_peered":"2024-09-22T01:15:43.204539+0000","last_clean":"2024-09-22T01:15:43.204539+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:15:43.204539+0000","last_undegraded":"2024-09-22T01:15:43.204539+0000","last_fullsized":"2024-09-22T01:15:43.204539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:06.730037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":31911709,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":13501,"num_read_kb":24099,"num_write":292,"num_write_kb":9584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"105'472","reported_seq":4416,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.557766+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:15:43.557766+0000","last_peered":"2024-09-22T01:15:43.557766+0000","last_clean":"2024-09-22T01:15:43.557766+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:15:43.557766+0000","last_undegraded":"2024-09-22T01:15:43.557766+0000","last_fullsized":"2024-09-22T01:15:43.557766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:55:53.401122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":48416139,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":20,"num_read":5105,"num_read_kb":22376,"num_write":582,"num_write_kb":12492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"104'241","reported_seq":2704,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.013906+0000","last_change":"2024-09-22T01:15:32.944791+0000","last_active":"2024-09-22T01:15:42.013906+0000","last_peered":"2024-09-22T01:15:42.013906+0000","last_clean":"2024-09-22T01:15:42.013906+0000","last_became_active":"2024-09-22T01:15:32.940539+0000","last_became_peered":"2024-09-22T01:15:32.940539+0000","last_unstale":"2024-09-22T01:15:42.013906+0000","last_undegraded":"2024-09-22T01:15:42.013906+0000","last_fullsized":"2024-09-22T01:15:42.013906+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:23.280647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":50907626,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":23,"num_read":2632,"num_read_kb":7585,"num_write":212,"num_write_kb":4267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"105'948","reported_seq":14288,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.335192+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:15:43.335192+0000","last_peered":"2024-09-22T01:15:43.335192+0000","last_clean":"2024-09-22T01:15:43.335192+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:15:43.335192+0000","last_undegraded":"2024-09-22T01:15:43.335192+0000","last_fullsized":"2024-09-22T01:15:43.335192+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:46:15.000880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":68265803,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":18,"num_read":17349,"num_read_kb":74988,"num_write":1557,"num_write_kb":26377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"105'294","reported_seq":4276,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.519457+0000","last_change":"2024-09-22T01:15:32.938982+0000","last_active":"2024-09-22T01:15:43.519457+0000","last_peered":"2024-09-22T01:15:43.519457+0000","last_clean":"2024-09-22T01:15:43.519457+0000","last_became_active":"2024-09-22T01:15:32.936966+0000","last_became_peered":"2024-09-22T01:15:32.936966+0000","last_unstale":"2024-09-22T01:15:43.519457+0000","last_undegraded":"2024-09-22T01:15:43.519457+0000","last_fullsized":"2024-09-22T01:15:43.519457+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:23:29.158081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":63704915,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":5299,"num_read_kb":25751,"num_write":270,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"105'243","reported_seq":4711,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.296955+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:15:43.296955+0000","last_peered":"2024-09-22T01:15:43.296955+0000","last_clean":"2024-09-22T01:15:43.296955+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:15:43.296955+0000","last_undegraded":"2024-09-22T01:15:43.296955+0000","last_fullsized":"2024-09-22T01:15:43.296955+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:23.653601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":52002410,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":4585,"num_read_kb":37483,"num_write":243,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"105'350","reported_seq":3593,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.348406+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:15:43.348406+0000","last_peered":"2024-09-22T01:15:43.348406+0000","last_clean":"2024-09-22T01:15:43.348406+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:15:43.348406+0000","last_undegraded":"2024-09-22T01:15:43.348406+0000","last_fullsized":"2024-09-22T01:15:43.348406+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:58:18.121857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":67247509,"num_objects":61,"num_object_clones":19,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":4133,"num_read_kb":32002,"num_write":374,"num_write_kb":11548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"105'348","reported_seq":10929,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.312299+0000","last_change":"2024-09-22T01:14:41.577883+0000","last_active":"2024-09-22T01:15:43.312299+0000","last_peered":"2024-09-22T01:15:43.312299+0000","last_clean":"2024-09-22T01:15:43.312299+0000","last_became_active":"2024-09-22T01:14:41.576276+0000","last_became_peered":"2024-09-22T01:14:41.576276+0000","last_unstale":"2024-09-22T01:15:43.312299+0000","last_undegraded":"2024-09-22T01:15:43.312299+0000","last_fullsized":"2024-09-22T01:15:43.312299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:31:07.544940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":47229034,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":23,"num_read":10610,"num_read_kb":24970,"num_write":398,"num_write_kb":12381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4954128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"104'398","reported_seq":1914,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.554512+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:15:42.554512+0000","last_peered":"2024-09-22T01:15:42.554512+0000","last_clean":"2024-09-22T01:15:42.554512+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:15:42.554512+0000","last_undegraded":"2024-09-22T01:15:42.554512+0000","last_fullsized":"2024-09-22T01:15:42.554512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:52.641561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":21637909,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":2685,"num_read_kb":15219,"num_write":548,"num_write_kb":3347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"105'583","reported_seq":2244,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.358211+0000","last_change":"2024-09-22T01:14:41.579663+0000","last_active":"2024-09-22T01:15:43.358211+0000","last_peered":"2024-09-22T01:15:43.358211+0000","last_clean":"2024-09-22T01:15:43.358211+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:15:43.358211+0000","last_undegraded":"2024-09-22T01:15:43.358211+0000","last_fullsized":"2024-09-22T01:15:43.358211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:56:45.547334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":65348068,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":4138,"num_read_kb":19680,"num_write":802,"num_write_kb":11534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014337+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:15:43.014337+0000","last_peered":"2024-09-22T01:15:43.014337+0000","last_clean":"2024-09-22T01:15:43.014337+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:15:43.014337+0000","last_undegraded":"2024-09-22T01:15:43.014337+0000","last_fullsized":"2024-09-22T01:15:43.014337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"105'398","reported_seq":2940,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014118+0000","last_change":"2024-09-22T01:15:32.938608+0000","last_active":"2024-09-22T01:15:43.014118+0000","last_peered":"2024-09-22T01:15:43.014118+0000","last_clean":"2024-09-22T01:15:43.014118+0000","last_became_active":"2024-09-22T01:15:32.937080+0000","last_became_peered":"2024-09-22T01:15:32.937080+0000","last_unstale":"2024-09-22T01:15:43.014118+0000","last_undegraded":"2024-09-22T01:15:43.014118+0000","last_fullsized":"2024-09-22T01:15:43.014118+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:12.539266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":35080637,"num_objects":46,"num_object_clones":14,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":5150,"num_read_kb":39840,"num_write":377,"num_write_kb":22695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":28,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.014055+0000","last_change":"2024-09-22T01:15:32.938118+0000","last_active":"2024-09-22T01:15:42.014055+0000","last_peered":"2024-09-22T01:15:42.014055+0000","last_clean":"2024-09-22T01:15:42.014055+0000","last_became_active":"2024-09-22T01:15:32.937355+0000","last_became_peered":"2024-09-22T01:15:32.937355+0000","last_unstale":"2024-09-22T01:15:42.014055+0000","last_undegraded":"2024-09-22T01:15:42.014055+0000","last_fullsized":"2024-09-22T01:15:42.014055+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:32.494366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"105'228","reported_seq":2127,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.371867+0000","last_change":"2024-09-22T01:15:32.928350+0000","last_active":"2024-09-22T01:15:43.371867+0000","last_peered":"2024-09-22T01:15:43.371867+0000","last_clean":"2024-09-22T01:15:43.371867+0000","last_became_active":"2024-09-22T01:15:32.926099+0000","last_became_peered":"2024-09-22T01:15:32.926099+0000","last_unstale":"2024-09-22T01:15:43.371867+0000","last_undegraded":"2024-09-22T01:15:43.371867+0000","last_fullsized":"2024-09-22T01:15:43.371867+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:07:04.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":34095983,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":21,"num_read":2337,"num_read_kb":24321,"num_write":218,"num_write_kb":9788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014520+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:15:43.014520+0000","last_peered":"2024-09-22T01:15:43.014520+0000","last_clean":"2024-09-22T01:15:43.014520+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:15:43.014520+0000","last_undegraded":"2024-09-22T01:15:43.014520+0000","last_fullsized":"2024-09-22T01:15:43.014520+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"105'479","reported_seq":2324,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.030485+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:15:43.030485+0000","last_peered":"2024-09-22T01:15:43.030485+0000","last_clean":"2024-09-22T01:15:43.030485+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:15:43.030485+0000","last_undegraded":"2024-09-22T01:15:43.030485+0000","last_fullsized":"2024-09-22T01:15:43.030485+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:23.860202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":29917963,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":25,"num_read":4402,"num_read_kb":18549,"num_write":664,"num_write_kb":6139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":181,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.026884+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:15:43.026884+0000","last_peered":"2024-09-22T01:15:43.026884+0000","last_clean":"2024-09-22T01:15:43.026884+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:15:43.026884+0000","last_undegraded":"2024-09-22T01:15:43.026884+0000","last_fullsized":"2024-09-22T01:15:43.026884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"105'466","reported_seq":3218,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.584732+0000","last_change":"2024-09-22T01:14:41.570935+0000","last_active":"2024-09-22T01:15:43.584732+0000","last_peered":"2024-09-22T01:15:43.584732+0000","last_clean":"2024-09-22T01:15:43.584732+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:15:43.584732+0000","last_undegraded":"2024-09-22T01:15:43.584732+0000","last_fullsized":"2024-09-22T01:15:43.584732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:24.829918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":43247794,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":4075,"num_read_kb":29894,"num_write":612,"num_write_kb":10687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"104'504","reported_seq":3130,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.243185+0000","last_change":"2024-09-22T01:14:38.704568+0000","last_active":"2024-09-22T01:15:42.243185+0000","last_peered":"2024-09-22T01:15:42.243185+0000","last_clean":"2024-09-22T01:15:42.243185+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:15:42.243185+0000","last_undegraded":"2024-09-22T01:15:42.243185+0000","last_fullsized":"2024-09-22T01:15:42.243185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:39.006012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":35415539,"num_objects":75,"num_object_clones":22,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":23,"num_read":3867,"num_read_kb":25426,"num_write":636,"num_write_kb":10113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":166,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014475+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:15:43.014475+0000","last_peered":"2024-09-22T01:15:43.014475+0000","last_clean":"2024-09-22T01:15:43.014475+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:15:43.014475+0000","last_undegraded":"2024-09-22T01:15:43.014475+0000","last_fullsized":"2024-09-22T01:15:43.014475+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"105'341","reported_seq":2016,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.176886+0000","last_change":"2024-09-22T01:14:06.517430+0000","last_active":"2024-09-22T01:15:43.176886+0000","last_peered":"2024-09-22T01:15:43.176886+0000","last_clean":"2024-09-22T01:15:43.176886+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:15:43.176886+0000","last_undegraded":"2024-09-22T01:15:43.176886+0000","last_fullsized":"2024-09-22T01:15:43.176886+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:38:25.066856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":38351373,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":2657,"num_read_kb":25711,"num_write":386,"num_write_kb":7606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014123+0000","last_change":"2024-09-22T01:13:57.404403+0000","last_active":"2024-09-22T01:15:43.014123+0000","last_peered":"2024-09-22T01:15:43.014123+0000","last_clean":"2024-09-22T01:15:43.014123+0000","last_became_active":"2024-09-22T01:13:55.013743+0000","last_became_peered":"2024-09-22T01:13:55.013743+0000","last_unstale":"2024-09-22T01:15:43.014123+0000","last_undegraded":"2024-09-22T01:15:43.014123+0000","last_fullsized":"2024-09-22T01:15:43.014123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:55:36.318022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075554500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"105'347","reported_seq":1859,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204699+0000","last_change":"2024-09-22T01:15:32.940653+0000","last_active":"2024-09-22T01:15:43.204699+0000","last_peered":"2024-09-22T01:15:43.204699+0000","last_clean":"2024-09-22T01:15:43.204699+0000","last_became_active":"2024-09-22T01:15:32.937217+0000","last_became_peered":"2024-09-22T01:15:32.937217+0000","last_unstale":"2024-09-22T01:15:43.204699+0000","last_undegraded":"2024-09-22T01:15:43.204699+0000","last_fullsized":"2024-09-22T01:15:43.204699+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:16:18.263018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":28367676,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":3012,"num_read_kb":7898,"num_write":372,"num_write_kb":9158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014071+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:15:43.014071+0000","last_peered":"2024-09-22T01:15:43.014071+0000","last_clean":"2024-09-22T01:15:43.014071+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:15:43.014071+0000","last_undegraded":"2024-09-22T01:15:43.014071+0000","last_fullsized":"2024-09-22T01:15:43.014071+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"105'439","reported_seq":5261,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.031926+0000","last_change":"2024-09-22T01:15:32.932715+0000","last_active":"2024-09-22T01:15:43.031926+0000","last_peered":"2024-09-22T01:15:43.031926+0000","last_clean":"2024-09-22T01:15:43.031926+0000","last_became_active":"2024-09-22T01:15:32.929890+0000","last_became_peered":"2024-09-22T01:15:32.929890+0000","last_unstale":"2024-09-22T01:15:43.031926+0000","last_undegraded":"2024-09-22T01:15:43.031926+0000","last_fullsized":"2024-09-22T01:15:43.031926+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:17:34.222484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":85750894,"num_objects":62,"num_object_clones":16,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":19,"num_read":6922,"num_read_kb":24445,"num_write":511,"num_write_kb":9638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":184,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.016319+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:15:43.016319+0000","last_peered":"2024-09-22T01:15:43.016319+0000","last_clean":"2024-09-22T01:15:43.016319+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:15:43.016319+0000","last_undegraded":"2024-09-22T01:15:43.016319+0000","last_fullsized":"2024-09-22T01:15:43.016319+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":333,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.024730+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:15:43.024730+0000","last_peered":"2024-09-22T01:15:43.024730+0000","last_clean":"2024-09-22T01:15:43.024730+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:15:43.024730+0000","last_undegraded":"2024-09-22T01:15:43.024730+0000","last_fullsized":"2024-09-22T01:15:43.024730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"105'322","reported_seq":1682,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.307396+0000","last_change":"2024-09-22T01:15:32.939125+0000","last_active":"2024-09-22T01:15:43.307396+0000","last_peered":"2024-09-22T01:15:43.307396+0000","last_clean":"2024-09-22T01:15:43.307396+0000","last_became_active":"2024-09-22T01:15:32.937085+0000","last_became_peered":"2024-09-22T01:15:32.937085+0000","last_unstale":"2024-09-22T01:15:43.307396+0000","last_undegraded":"2024-09-22T01:15:43.307396+0000","last_fullsized":"2024-09-22T01:15:43.307396+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:42:54.725390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":63093377,"num_objects":66,"num_object_clones":23,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":22,"num_read":1962,"num_read_kb":14598,"num_write":266,"num_write_kb":3876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"105'750","reported_seq":16569,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.133806+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:15:43.133806+0000","last_peered":"2024-09-22T01:15:43.133806+0000","last_clean":"2024-09-22T01:15:43.133806+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:15:43.133806+0000","last_undegraded":"2024-09-22T01:15:43.133806+0000","last_fullsized":"2024-09-22T01:15:43.133806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:44.800290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":93056327,"num_objects":81,"num_object_clones":21,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":24,"num_read":21212,"num_read_kb":78250,"num_write":802,"num_write_kb":22521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"105'395","reported_seq":2577,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.014443+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:15:43.014443+0000","last_peered":"2024-09-22T01:15:43.014443+0000","last_clean":"2024-09-22T01:15:43.014443+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:15:43.014443+0000","last_undegraded":"2024-09-22T01:15:43.014443+0000","last_fullsized":"2024-09-22T01:15:43.014443+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:50:35.904065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":27862245,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":5186,"num_read_kb":9876,"num_write":387,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"105'404","reported_seq":3005,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.187255+0000","last_change":"2024-09-22T01:14:06.538578+0000","last_active":"2024-09-22T01:15:43.187255+0000","last_peered":"2024-09-22T01:15:43.187255+0000","last_clean":"2024-09-22T01:15:43.187255+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:15:43.187255+0000","last_undegraded":"2024-09-22T01:15:43.187255+0000","last_fullsized":"2024-09-22T01:15:43.187255+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:54:55.321982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":42504998,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":4098,"num_read_kb":16291,"num_write":548,"num_write_kb":8521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"104'509","reported_seq":3015,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.013977+0000","last_change":"2024-09-22T01:14:06.521861+0000","last_active":"2024-09-22T01:15:42.013977+0000","last_peered":"2024-09-22T01:15:42.013977+0000","last_clean":"2024-09-22T01:15:42.013977+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:15:42.013977+0000","last_undegraded":"2024-09-22T01:15:42.013977+0000","last_fullsized":"2024-09-22T01:15:42.013977+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:21:54.089887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":35868587,"num_objects":84,"num_object_clones":29,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":34,"num_read":3503,"num_read_kb":26065,"num_write":634,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"105'542","reported_seq":1469,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.262783+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:15:43.262783+0000","last_peered":"2024-09-22T01:15:43.262783+0000","last_clean":"2024-09-22T01:15:43.262783+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:15:43.262783+0000","last_undegraded":"2024-09-22T01:15:43.262783+0000","last_fullsized":"2024-09-22T01:15:43.262783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:28.428817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":22060365,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":15,"num_read":2867,"num_read_kb":11363,"num_write":769,"num_write_kb":6282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"105'696","reported_seq":2773,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.185119+0000","last_change":"2024-09-22T01:14:41.579603+0000","last_active":"2024-09-22T01:15:43.185119+0000","last_peered":"2024-09-22T01:15:43.185119+0000","last_clean":"2024-09-22T01:15:43.185119+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:15:43.185119+0000","last_undegraded":"2024-09-22T01:15:43.185119+0000","last_fullsized":"2024-09-22T01:15:43.185119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:29:37.649550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":34397159,"num_objects":68,"num_object_clones":18,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":22,"num_read":4429,"num_read_kb":20364,"num_write":962,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"104'477","reported_seq":1241,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-22T01:15:42.014032+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:15:42.014032+0000","last_peered":"2024-09-22T01:15:42.014032+0000","last_clean":"2024-09-22T01:15:42.014032+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:15:42.014032+0000","last_undegraded":"2024-09-22T01:15:42.014032+0000","last_fullsized":"2024-09-22T01:15:42.014032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:41:44.236986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":32194018,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":24,"num_read":1988,"num_read_kb":19901,"num_write":643,"num_write_kb":10263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"105'454","reported_seq":2289,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.204447+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:15:43.204447+0000","last_peered":"2024-09-22T01:15:43.204447+0000","last_clean":"2024-09-22T01:15:43.204447+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:15:43.204447+0000","last_undegraded":"2024-09-22T01:15:43.204447+0000","last_fullsized":"2024-09-22T01:15:43.204447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:04:45.370302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":34812101,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":20,"num_read":3569,"num_read_kb":14016,"num_write":555,"num_write_kb":4152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"105'726","reported_seq":3558,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.565672+0000","last_change":"2024-09-22T01:14:41.582658+0000","last_active":"2024-09-22T01:15:43.565672+0000","last_peered":"2024-09-22T01:15:43.565672+0000","last_clean":"2024-09-22T01:15:43.565672+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:15:43.565672+0000","last_undegraded":"2024-09-22T01:15:43.565672+0000","last_fullsized":"2024-09-22T01:15:43.565672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:44:40.979721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":47346680,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":13,"num_read":6271,"num_read_kb":48929,"num_write":979,"num_write_kb":22025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"105'618","reported_seq":3086,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.244811+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:15:43.244811+0000","last_peered":"2024-09-22T01:15:43.244811+0000","last_clean":"2024-09-22T01:15:43.244811+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:15:43.244811+0000","last_undegraded":"2024-09-22T01:15:43.244811+0000","last_fullsized":"2024-09-22T01:15:43.244811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:10:06.760100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":57271130,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":22,"num_read":4829,"num_read_kb":24310,"num_write":814,"num_write_kb":10292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"105'517","reported_seq":2711,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-22T01:15:43.369541+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:15:43.369541+0000","last_peered":"2024-09-22T01:15:43.369541+0000","last_clean":"2024-09-22T01:15:43.369541+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:15:43.369541+0000","last_undegraded":"2024-09-22T01:15:43.369541+0000","last_fullsized":"2024-09-22T01:15:43.369541+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:36:45.318920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":58759511,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":26,"num_read":4493,"num_read_kb":29013,"num_write":614,"num_write_kb":5882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1468108745,"num_objects":1933,"num_object_clones":513,"num_object_copies":3866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1933,"num_whiteouts":649,"num_read":179617,"num_read_kb":848326,"num_write":17683,"num_write_kb":339695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":46033956,"num_keys_recovered":155,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":481640448,"data_stored":660597812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14352,"ondisk_log_size":14352,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79144,"kb_used_data":43792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94292696,"statfs":{"total":96636764160,"available":96555720704,"internally_reserved":0,"allocated":44843008,"data_stored":55912598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83344,"kb_used_data":43840,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94288496,"statfs":{"total":96636764160,"available":96551419904,"internally_reserved":0,"allocated":44892160,"data_stored":55467226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111516,"kb_used_data":67840,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94260324,"statfs":{"total":96636764160,"available":96522571776,"internally_reserved":0,"allocated":69468160,"data_stored":91506573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":107000,"kb_used_data":67488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94264840,"statfs":{"total":96636764160,"available":96527196160,"internally_reserved":0,"allocated":69107712,"data_stored":97500959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123028,"kb_used_data":73756,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94248812,"statfs":{"total":96636764160,"available":96510783488,"internally_reserved":0,"allocated":75526144,"data_stored":106058192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119608,"kb_used_data":70324,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94252232,"statfs":{"total":96636764160,"available":96514285568,"internally_reserved":0,"allocated":72011776,"data_stored":97976351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105368,"kb_used_data":60296,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94266472,"statfs":{"total":96636764160,"available":96528867328,"internally_reserved":0,"allocated":61743104,"data_stored":86210706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98548,"kb_used_data":53476,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94273292,"statfs":{"total":96636764160,"available":96535851008,"internally_reserved":0,"allocated":54759424,"data_stored":75478033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53567488,"data_stored":74925375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60026880,"data_stored":85129593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":70836224,"data_stored":97435662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":74366976,"data_stored":105537461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67891200,"data_stored":96944923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":67637248,"data_stored":90348280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43712512,"data_stored":54940122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43601920,"data_stored":55336396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:15:45.916 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:15:45.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:45.917 INFO:tasks.thrashosds.thrasher:Reweighting osd 7 to 0.2866473969068815 2024-09-22T01:15:45.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 0.2866473969068815 2024-09-22T01:15:46.003 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:46.001+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:46.032 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:46.032 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:46.032 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:46.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:46.049 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:15:46.132 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone49 from 0xbb67ca4 (overlap 0x92b040d) to 0x66ae93c 2024-09-22T01:15:46.134 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:46.133+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:46.204 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:46.204+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:46.210 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:46.211 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:46.211 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:46.225 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:46.305 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:46.304+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.243903951, 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.243804011000002, 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:46.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:46.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:46.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.243735119999997, 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:46.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.243659057000002, 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:46.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:46.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:46.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:46.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:46.412 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.243596269000001, 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:46.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:46.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242965335000001, 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242936196000002, 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.421 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:46.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242906871000002, 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:46.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:46.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242876234999997, 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242840567000002, 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242811584000002, 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:46.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.440 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242777371999999, 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:46.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.444 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242708552000003, 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:46.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.448 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242608038, 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:46.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:46.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242539884000003, 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:46.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:46.456 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242456154000003, 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.457 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:46.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:46.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242322276000003, 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:46.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:46.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242195426999999, 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242158472, 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:46.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:46.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.242106829999997, 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:46.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:46.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.477 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:46.478 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-22T01:15:46.478 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:46.478 INFO:teuthology.orchestra.run.smithi096.stdout:5763 writesame 0x9418f68 thru 0x942782e data_size 0x1(0xe8c7 bytes) 2024-09-22T01:15:46.478 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:46.476+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:46.479 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:46.579 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:46.576+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:46.605 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:46.606 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:46.606 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:46.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:46.679 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:46.677+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:46.762 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:46.762 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:46.762 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:46.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:46.822 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:46.821+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:46.881 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:46.880+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.649007393000005, 2024-09-22T01:15:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.648947399999997, 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.648874907000007, 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.648806594999996, 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.648748671000007, 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:46.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:46.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 90.648692781999998, 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:46.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:46.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:46.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 76.761748005000001, 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:46.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.674256013999994, 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:47.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.961310252999994, 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:47.005 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:47.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 70.800913260000002, 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:47.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:47.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:47.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:47.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:47.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122050 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2850816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.556986542000001, 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016781720999999999, 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.016 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122050 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405140+0000", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405142+0000", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1220000000000002e-06 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405148+0000", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2110000000000003e-06 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405153+0000", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421546+0000", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016393557 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421632+0000", 2024-09-22T01:15:47.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5611000000000004e-05 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421921+0000", 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028914000000000001 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122051 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2916352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.556965959999999, 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017012590000000001, 2024-09-22T01:15:47.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122051 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.82e-07 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405163+0000", 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2390000000000001e-06 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405169+0000", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6050000000000003e-06 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405191+0000", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1625999999999999e-05 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421958+0000", 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016766873000000002 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422046+0000", 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8041000000000006e-05 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422172+0000", 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012672399999999999 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122052 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2981888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.556851387, 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017140268, 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122052 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405276+0000", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.874e-06 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405290+0000", 2024-09-22T01:15:47.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3325e-05 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405319+0000", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9791999999999999e-05 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405338+0000", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9004999999999998e-05 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422212+0000", 2024-09-22T01:15:47.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016874005000000001 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422295+0000", 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2891000000000003e-05 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422415+0000", 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119376 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122230 3.4 3:235bf1fd:::rbd_data.1124ddce6057.0000000000000002:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.553368842999999, 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01649316, 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122230 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408758+0000", 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-22T01:15:47.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408763+0000", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1760000000000001e-06 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408790+0000", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6855e-05 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408807+0000", 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7152000000000001e-05 2024-09-22T01:15:47.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425125+0000", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016317941999999998 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425176+0000", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1263e-05 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425250+0000", 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3621999999999994e-05 2024-09-22T01:15:47.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122241 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2801664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552753767, 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016601057999999998, 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122241 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409373+0000", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2270000000000001e-06 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409379+0000", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409396+0000", 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6985000000000001e-05 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409408+0000", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2819e-05 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425570+0000", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016161983000000001 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425671+0000", 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010096 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425973+0000", 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030178699999999998 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122242 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552701756999999, 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01685275, 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122242 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409425+0000", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409430+0000", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599999999999998e-06 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409447+0000", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409459+0000", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2672e-05 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426007+0000", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016547671 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426065+0000", 2024-09-22T01:15:47.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7652000000000003e-05 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426277+0000", 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021208300000000001 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122243 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2932736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552650766999999, 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017069536, 2024-09-22T01:15:47.041 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122243 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409476+0000", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409481+0000", 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409498+0000", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409511+0000", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426325+0000", 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016814030000000001 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426439+0000", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011437700000000001 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426545+0000", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105628 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122244 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:47.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552599311, 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017170747, 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122244 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409527+0000", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409532+0000", 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8130000000000003e-06 2024-09-22T01:15:47.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409549+0000", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6575999999999999e-05 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409562+0000", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2772000000000001e-05 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426577+0000", 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017014945 2024-09-22T01:15:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426631+0000", 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4537999999999999e-05 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426697+0000", 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6075999999999996e-05 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.049 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122245 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552548154, 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017515171, 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122245 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409579+0000", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.026e-06 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409583+0000", 2024-09-22T01:15:47.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6829999999999999e-06 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409646+0000", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2803000000000002e-05 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409662+0000", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5418999999999999e-05 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426742+0000", 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017080346 2024-09-22T01:15:47.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426830+0000", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8527000000000001e-05 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427093+0000", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000262367 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122246 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:47.053 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 10.552428696, 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017546235, 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122246 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409698+0000", 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.283e-06 2024-09-22T01:15:47.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409713+0000", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4839e-05 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409729+0000", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5931999999999999e-05 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409796+0000", 2024-09-22T01:15:47.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7149000000000004e-05 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427115+0000", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017319068 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427166+0000", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0784e-05 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427243+0000", 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7180000000000003e-05 2024-09-22T01:15:47.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.058 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.058 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.058 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:47.059 INFO:teuthology.orchestra.run.smithi096.stdout:5764 trunc from 0xbb67ca4 to 0xb98f52b 2024-09-22T01:15:47.059 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.7 to 0.286647 (4961) 2024-09-22T01:15:47.060 INFO:teuthology.orchestra.run.smithi096.stdout:5765 write 0x737d594 thru 0x7381765 (0x41d2 bytes) 2024-09-22T01:15:47.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:47.060 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:47.057+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:47.160 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:47.160+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:47.234 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:47.234 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:47.234 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:47.234 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:47.248 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:47.294 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:47.293+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:47.362 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:47.360+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:47.367 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:47.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:47.367 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:47.367 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:47.367 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:47.381 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:47.463 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:47.460+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:47.549 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038900691, 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:47.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:47.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038842061, 2024-09-22T01:15:47.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:15:47.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:15:47.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038819181, 2024-09-22T01:15:47.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:15:47.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:47.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0388011180000001, 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:47.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0387636309999999, 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038738009, 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0387171610000001, 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0386999219999999, 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:15:47.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038647828, 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.583 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:47.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0386279549999999, 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:15:47.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.590 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0385817070000001, 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.591 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0385610199999999, 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:15:47.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:47.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:47.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038512331, 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.599 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:47.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:47.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:15:47.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0384432779999999, 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:15:47.604 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:15:47.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:15:47.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:47.608 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038423444, 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:47.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:47.612 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0383748129999999, 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:15:47.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:15:47.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038353753, 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:15:47.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0383031899999999, 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.0382822629999999, 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:15:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:15:47.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 1.038254376, 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:47.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.632 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:47.632 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:47.632 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:47.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:47.633 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:47.629+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:47.634 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:47.734 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:47.732+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:47.765 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:47.765 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:147228 3.c 3.86c9450c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:47.353032+0000", 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.41253369200000001, 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.412556329, 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 147228 2024-09-22T01:15:47.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353032+0000", 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353032+0000", 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:47.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353034+0000", 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.68e-06 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353042+0000", 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4910000000000005e-06 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353060+0000", 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8209000000000001e-05 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353068+0000", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353147+0000", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8576000000000001e-05 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:47.353161+0000", 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4297999999999999e-05 2024-09-22T01:15:47.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-22T01:15:47.770 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:47.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:47.788 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:47.788 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:47.806 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:47.806 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:47.821 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:47.822 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:47.835 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:47.833+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:47.838 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:47.838 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:47.860 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:47.861 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:47.881 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:47.881 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:47.901 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:47.901 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:47.902 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:47.902 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:47.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:47.923 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:47.923 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:47.936 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:47.933+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:48.037 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:48.036+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:48.051 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.051 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:48.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:15:48.138 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:48.137+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:48.229 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.229 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.239 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:48.237+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:48.245 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:48.341 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:48.340+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:48.359 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.359 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.373 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:48.442 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:48.437+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:48.540 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.541 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.542 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:48.537+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:48.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:48.647 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:48.645+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:48.678 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.678 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:48.744 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:48.744+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:48.816 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.816 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:48.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:48.873 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:48.872+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:48.946 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:48.941+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:48.989 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:48.989 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:49.047 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:49.045+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:49.049 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:15:49.136 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:49.136 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.147 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:49.144+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:49.151 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:49.248 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:49.248+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:49.336 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:49.336 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.349 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:49.345+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:49.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:49.450 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:49.445+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:49.536 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:49.537 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.540 INFO:teuthology.orchestra.run.smithi096.stdout:5767 read 0x5589e56 thru 0x5594b01 (0xacac bytes) 2024-09-22T01:15:49.541 INFO:teuthology.orchestra.run.smithi096.stdout:5768 punch from 0x48ef846 to 0x48f9f4e, (0xa708 bytes) 2024-09-22T01:15:49.541 INFO:teuthology.orchestra.run.smithi096.stdout:5769 punch from 0x60b36bf to 0x60b9314, (0x5c55 bytes) 2024-09-22T01:15:49.541 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:49.541 INFO:teuthology.orchestra.run.smithi096.stdout:5771 write 0x33428c7 thru 0x3343f51 (0x168b bytes) 2024-09-22T01:15:49.551 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:49.548+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:49.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:49.558 INFO:teuthology.orchestra.run.smithi096.stdout:5772 write 0x63bd17a thru 0x63c5ea9 (0x8d30 bytes) 2024-09-22T01:15:49.652 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:49.649+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:49.725 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:49.725 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:49.753 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:49.752+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:49.853 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:49.849+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:49.937 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:49.938 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:49.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:49.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:49.994 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:49.993+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:50.055 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:50.052+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:50.102 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:50.102 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:50.107 INFO:teuthology.orchestra.run.smithi096.stdout:5773 read 0x2489466 thru 0x2491e42 (0x89dd bytes) 2024-09-22T01:15:50.110 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:50.110 INFO:teuthology.orchestra.run.smithi096.stdout:5776 punch from 0x8adb478 to 0x8ae6dd3, (0xb95b bytes) 2024-09-22T01:15:50.110 INFO:teuthology.orchestra.run.smithi096.stdout:5778 write 0xaaaf166 thru 0xaab03a3 (0x123e bytes) 2024-09-22T01:15:50.119 INFO:teuthology.orchestra.run.smithi096.stdout:5779 write 0x5bd665e thru 0x5be51fe (0xeba1 bytes) 2024-09-22T01:15:50.122 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:50.123 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:50.124 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:50.124 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:50.141 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:50.145 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:50.145 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:50.155 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:50.152+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:50.162 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:50.162 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:50.180 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:50.181 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:50.199 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:50.199 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:50.215 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:50.216 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:50.233 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:50.234 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:50.258 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:50.258 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:50.258 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:50.273 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:50.296 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:50.296+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:50.357 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:50.356+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:50.395 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:50.395 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:50.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:50.458 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:50.456+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:50.538 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.391441499999999, 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:50.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:15:50.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.543 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.391341560000001, 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.544 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.391272669000003, 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:15:50.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.391196606000001, 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.391133818, 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:15:50.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390502884, 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:15:50.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390473745000001, 2024-09-22T01:15:50.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:15:50.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:15:50.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390444420000001, 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:50.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:15:50.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:15:50.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:50.574 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390413784000003, 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:50.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:15:50.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.580 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390378116000001, 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390349133000001, 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.585 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:15:50.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:15:50.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390314920999998, 2024-09-22T01:15:50.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.593 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:50.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:15:50.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.600 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390246101000002, 2024-09-22T01:15:50.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.602 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:50.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:15:50.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390145586999999, 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:15:50.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:15:50.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.390077433000002, 2024-09-22T01:15:50.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:50.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:15:50.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.389993703000002, 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:15:50.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:50.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.389859825000002, 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:50.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.624 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.389732975999998, 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:15:50.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:15:50.628 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.389696020999999, 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.629 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:15:50.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 41.389644379000003, 2024-09-22T01:15:50.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:50.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:15:50.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.637 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.638 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.638 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:50.638 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:50.638 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:50.639 INFO:teuthology.orchestra.run.smithi096.stdout:5781 writesame 0xa71d359 thru 0xa72c0b1 data_size 0x1(0xed59 bytes) 2024-09-22T01:15:50.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:50.640 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:50.637+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:50.741 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:50.740+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:50.812 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:50.813 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:50.813 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:50.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:50.842 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:50.840+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:50.943 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:50.940+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:50.949 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:50.949 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:50.950 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:50.950 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:50.950 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:50.963 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:51.043 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:51.040+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:51.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:15:51.084 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7228 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.768227589000006, 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016585506, 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.085 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7228 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313118+0000", 2024-09-22T01:15:51.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313119+0000", 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1069999999999999e-06 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313126+0000", 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3330000000000002e-06 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313144+0000", 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8337000000000001e-05 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313158+0000", 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3703000000000001e-05 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329290+0000", 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016131791999999999 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329507+0000", 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021706199999999999 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329704+0000", 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019717199999999999 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.089 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7229 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.768167595999998, 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017176845, 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7229 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313178+0000", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313180+0000", 2024-09-22T01:15:51.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.19e-06 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313188+0000", 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7630000000000006e-06 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313206+0000", 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7697e-05 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313221+0000", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5136e-05 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.329874+0000", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016652831999999999 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330140+0000", 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026604500000000002 2024-09-22T01:15:51.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330355+0000", 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021518199999999999 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7230 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:51.094 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.768095102999993, 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017680634000000001, 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7230 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313251+0000", 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313252+0000", 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2899999999999999e-06 2024-09-22T01:15:51.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313259+0000", 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6069999999999996e-06 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313282+0000", 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3456e-05 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313299+0000", 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.66e-05 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330505+0000", 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017206181000000001 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330708+0000", 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000202609 2024-09-22T01:15:51.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.330931+0000", 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022389099999999999 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7231 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.768026790999997, 2024-09-22T01:15:51.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018109594999999999, 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7231 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:51.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313319+0000", 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313320+0000", 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1939999999999999e-06 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313326+0000", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6999999999999996e-06 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313345+0000", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8655999999999999e-05 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313358+0000", 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.342e-05 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331063+0000", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017704431 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331231+0000", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016810499999999999 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331429+0000", 2024-09-22T01:15:51.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00019808900000000001 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7232 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.767968866999993, 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018590780000000001, 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7232 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313377+0000", 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313379+0000", 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4559999999999999e-06 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313385+0000", 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9739999999999999e-06 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313402+0000", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7306e-05 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313415+0000", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3536999999999999e-05 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331514+0000", 2024-09-22T01:15:51.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.018098530000000002 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331702+0000", 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000187632 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.331968+0000", 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00026634500000000003 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.108 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:7233 3.4 3:27767290:::rbd_data.1124636e4501.000000000000000b:7 [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 94.767912977999998, 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019105726, 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 7233 2024-09-22T01:15:51.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313433+0000", 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313434+0000", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3400000000000001e-06 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313440+0000", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7459999999999997e-06 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313458+0000", 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8244e-05 2024-09-22T01:15:51.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.313472+0000", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3512999999999999e-05 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332087+0000", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01861486 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332340+0000", 2024-09-22T01:15:51.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00025347500000000002 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:16.332539+0000", 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000198548 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:21247 3.18 e47/24 3:1f821670:::rbd_data.11249c3a956.0000000000000002:head v 47'50, mlcod=45'48)", 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.880968201000002, 2024-09-22T01:15:51.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017361233, 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200378+0000", 2024-09-22T01:15:51.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.200380+0000", 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.013e-06 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203853+0000", 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0034735149999999999 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203905+0000", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2034999999999999e-05 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.203933+0000", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7246999999999998e-05 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204097+0000", 2024-09-22T01:15:51.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00016415200000000001 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.204146+0000", 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9254e-05 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217630+0000", 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.013484398 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:30.217739+0000", 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000108619 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_repop(client.4388.0:22363 3.5 e48/24 3:a16c3f53:::rbd_data.1124dfca90b7.0000000000000002:head v 48'36, mlcod=44'34)", 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 79.793476209999994, 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.019724206000000001, 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.118 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287870+0000", 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.287871+0000", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1310000000000001e-06 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292277+0000", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0044064600000000001 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292295+0000", 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7909000000000001e-05 2024-09-22T01:15:51.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292302+0000", 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0709999999999997e-06 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292359+0000", 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6777000000000002e-05 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.121 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.292375+0000", 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5896000000000002e-05 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307544+0000", 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.015168650000000001 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:31.307594+0000", 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0312000000000002e-05 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.122 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:23933 3.12 3:4b135163:::rbd_data.11246797a72b.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.080530448999994, 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024405647999999999, 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.123 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 23933 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000816+0000", 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.000817+0000", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8950000000000001e-06 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005781+0000", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0049632829999999998 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005805+0000", 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4644999999999999e-05 2024-09-22T01:15:51.125 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005817+0000", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1437000000000001e-05 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.005925+0000", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000107735 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006158+0000", 2024-09-22T01:15:51.126 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000233062 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 3", 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.006576+0000", 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00041860699999999999 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.021275+0000", 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.014698619 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:51.127 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.024977+0000", 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00370215 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025098+0000", 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000121536 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.025221+0000", 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000122679 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.128 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:24025 3.5 3:a1985718:::rbd_object_map.11246797a72b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 74.920133456000002, 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017782178999999999, 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.129 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 24025 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161213+0000", 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.130 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161215+0000", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0839999999999999e-06 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161232+0000", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7337999999999999e-05 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161260+0000", 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8081000000000001e-05 2024-09-22T01:15:51.131 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.161281+0000", 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1061e-05 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.170680+0000", 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0093993080000000003 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.171020+0000", 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00033921299999999999 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.174327+0000", 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0033074189999999998 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178931+0000", 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0046034919999999998 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "op_commit", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178938+0000", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8679999999999999e-06 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "commit_sent", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178972+0000", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3643000000000003e-05 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:14:36.178995+0000", 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2671999999999999e-05 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.134 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122050 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2850816~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.676206737999999, 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016781720999999999, 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122050 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405139+0000", 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405140+0000", 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405142+0000", 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1220000000000002e-06 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405148+0000", 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2110000000000003e-06 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405153+0000", 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421546+0000", 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016393557 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421632+0000", 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5611000000000004e-05 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421921+0000", 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00028914000000000001 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122051 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2916352~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.676186156, 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017012590000000001, 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122051 2024-09-22T01:15:51.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.141 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405160+0000", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.82e-07 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405163+0000", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2390000000000001e-06 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405169+0000", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6050000000000003e-06 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405191+0000", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1625999999999999e-05 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.421958+0000", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016766873000000002 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422046+0000", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8041000000000006e-05 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.144 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422172+0000", 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012672399999999999 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122052 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2981888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.676071583000001, 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017140268, 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.145 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122052 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405274+0000", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405276+0000", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.874e-06 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405290+0000", 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3325e-05 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405319+0000", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9791999999999999e-05 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.405338+0000", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9004999999999998e-05 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422212+0000", 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016874005000000001 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422295+0000", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2891000000000003e-05 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.422415+0000", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000119376 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122230 3.4 3:235bf1fd:::rbd_data.1124ddce6057.0000000000000002:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:51.149 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.672589039, 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01649316, 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122230 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:51.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408757+0000", 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408758+0000", 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.15e-06 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408763+0000", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1760000000000001e-06 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408790+0000", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6855e-05 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.408807+0000", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7152000000000001e-05 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425125+0000", 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016317941999999998 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425176+0000", 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1263e-05 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425250+0000", 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3621999999999994e-05 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122241 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2801664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671973962999999, 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016601057999999998, 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.154 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122241 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:51.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409372+0000", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409373+0000", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2270000000000001e-06 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409379+0000", 2024-09-22T01:15:51.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2970000000000003e-06 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409396+0000", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6985000000000001e-05 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409408+0000", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2819e-05 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425570+0000", 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016161983000000001 2024-09-22T01:15:51.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425671+0000", 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00010096 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.425973+0000", 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00030178699999999998 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.158 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122242 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2867200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671921953, 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.01685275, 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122242 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409424+0000", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409425+0000", 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.043e-06 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409430+0000", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0599999999999998e-06 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409447+0000", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6569e-05 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409459+0000", 2024-09-22T01:15:51.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2672e-05 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426007+0000", 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016547671 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426065+0000", 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7652000000000003e-05 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426277+0000", 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00021208300000000001 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122243 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2932736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671870963, 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017069536, 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.163 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122243 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409475+0000", 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409476+0000", 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0669999999999999e-06 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409481+0000", 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409498+0000", 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6642000000000001e-05 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409511+0000", 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2830000000000001e-05 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426325+0000", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.016814030000000001 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426439+0000", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011437700000000001 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426545+0000", 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000105628 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.167 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122244 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671819507, 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017170747, 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122244 2024-09-22T01:15:51.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409526+0000", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409527+0000", 2024-09-22T01:15:51.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0270000000000001e-06 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409532+0000", 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8130000000000003e-06 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409549+0000", 2024-09-22T01:15:51.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6575999999999999e-05 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409562+0000", 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2772000000000001e-05 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426577+0000", 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017014945 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426631+0000", 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4537999999999999e-05 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426697+0000", 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6075999999999996e-05 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122245 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3063808~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:51.172 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671768350000001, 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017515171, 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122245 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409578+0000", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409579+0000", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.026e-06 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409583+0000", 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6829999999999999e-06 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409646+0000", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2803000000000002e-05 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409662+0000", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5418999999999999e-05 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426742+0000", 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017080346 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.426830+0000", 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8527000000000001e-05 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427093+0000", 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000262367 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.176 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:122246 3.12 3:495d4763:::rbd_data.1124ddce6057.0000000000000003:2d [read 3129344~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.671648892, 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017546235, 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 122246 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:51.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409697+0000", 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409698+0000", 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.283e-06 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409713+0000", 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4839e-05 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409729+0000", 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5931999999999999e-05 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.409796+0000", 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7149000000000004e-05 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427115+0000", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.017319068 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427166+0000", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0784e-05 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:36.427243+0000", 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7180000000000003e-05 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.180 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.181 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.181 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.181 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:51.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:51.183 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:51.180+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:51.283 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:51.280+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:51.310 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:51.310 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:51.310 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:51.310 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:51.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:51.384 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:51.381+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:51.446 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:51.447 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:51.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:51.503 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:51.501+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:51.522 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:15:51.537 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:51.545 INFO:teuthology.orchestra.run.smithi096.stdout:5785 trunc from 0xb98f52b to 0x79c0749 2024-09-22T01:15:51.596 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:51.596+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1291361059999998, 2024-09-22T01:15:51.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:15:51.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:15:51.644 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.129077476, 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:51.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1290545959999996, 2024-09-22T01:15:51.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1290365329999998, 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.654 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:15:51.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.658 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1289990459999997, 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:15:51.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:15:51.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1289734239999998, 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:51.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:15:51.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:15:51.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1289525759999997, 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.667 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.668 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:15:51.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:51.671 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1289353369999997, 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:15:51.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:15:51.674 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1288832429999998, 2024-09-22T01:15:51.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:15:51.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:51.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1288633700000004, 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:15:51.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:51.683 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1288171220000001, 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:51.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:15:51.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.688 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.688 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:51.688 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1287964349999999, 2024-09-22T01:15:51.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:15:51.689 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.689 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.689 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.690 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.693 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1287477460000002, 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:15:51.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.695 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:15:51.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:15:51.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1286786930000003, 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.698 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.701 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1286588589999997, 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:15:51.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:15:51.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1286102280000003, 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.706 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:15:51.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:15:51.709 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1285891680000004, 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.710 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:15:51.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1285386050000001, 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:15:51.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:15:51.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1285176779999997, 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.721 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:15:51.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:15:51.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 5.1284897909999998, 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:51.725 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:51.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:51.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:15:51.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:51.729 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:51.730 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:51.729+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:51.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:51.831 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:51.828+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:51.900 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:51.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:51.900 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:51.900 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:51.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:51.932 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:51.929+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:52.032 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:52.032+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:52.084 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:52.084 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:52.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:15:52.087 INFO:teuthology.orchestra.run.smithi096.stdout:5787 trunc from 0x79c0749 to 0x38e88f 2024-09-22T01:15:52.104 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:52.104 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:52.104 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:52.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:52.136 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:52.129+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:52.235 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:52.232+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:52.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:52.335 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:52.332+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.344 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.398557904, 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:15:52.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:15:52.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:52.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.242708603000001, 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.242687274, 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:52.360 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.242224201000001, 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:15:52.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:15:52.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.242197331, 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:15:52.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:15:52.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.242173823, 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:15:52.369 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:15:52.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:15:52.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.241976877000001, 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:15:52.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:15:52.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.241948811, 2024-09-22T01:15:52.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:15:52.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240707371999999, 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:15:52.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:15:52.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240669606000001, 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:15:52.387 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:15:52.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240645652, 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:15:52.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240624565999999, 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240604102000001, 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:52.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.404 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240584081, 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240564882999999, 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:15:52.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240543771, 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:52.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240523179, 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240501818, 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.422 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:15:52.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:15:52.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:15:52.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240480504000001, 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:15:52.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:15:52.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.240461485999999, 2024-09-22T01:15:52.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:52.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:15:52.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:15:52.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:52.434 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:52.435 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:52.436 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:52.436 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:52.437 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:52.437 INFO:teuthology.orchestra.run.smithi096.stdout:5788 trunc from 0x38e88f to 0xdf3e94b 2024-09-22T01:15:52.437 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:52.432+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:52.437 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:52.438 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:52.438 INFO:teuthology.orchestra.run.smithi096.stdout:5789 read 0xc28f054 thru 0xc29d0a0 (0xe04d bytes) 2024-09-22T01:15:52.446 INFO:teuthology.orchestra.run.smithi096.stdout:5790 write 0xe67109b thru 0xe67e7ce (0xd734 bytes) 2024-09-22T01:15:52.460 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:52.460 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:52.466 INFO:teuthology.orchestra.run.smithi096.stdout:5791 punch from 0x887062a to 0x8874435, (0x3e0b bytes) 2024-09-22T01:15:52.467 INFO:teuthology.orchestra.run.smithi096.stdout:5792 write 0x9df294e thru 0x9dfcf4e (0xa601 bytes) 2024-09-22T01:15:52.489 INFO:teuthology.orchestra.run.smithi096.stdout:5794 writesame 0xaa5a0b thru 0xaab10a data_size 0x60(0x5700 bytes) 2024-09-22T01:15:52.491 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:52.491 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:52.508 INFO:teuthology.orchestra.run.smithi096.stdout:5795 write 0x9629128 thru 0x9637ba3 (0xea7c bytes) 2024-09-22T01:15:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:15:52.518 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:15:52.528 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:52.528 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:52.532 INFO:teuthology.orchestra.run.smithi096.stdout:5796 read 0xa8ba723 thru 0xa8c928a (0xeb68 bytes) 2024-09-22T01:15:52.533 INFO:teuthology.orchestra.run.smithi096.stdout:5797 read 0x5c8b1a6 thru 0x5c91779 (0x65d4 bytes) 2024-09-22T01:15:52.533 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.533 INFO:teuthology.orchestra.run.smithi096.stdout:5800 writesame 0x78ac8cf thru 0x78af0e4 data_size 0x2(0x2816 bytes) 2024-09-22T01:15:52.537 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:52.536+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:52.540 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-22T01:15:51.559386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1561878420,"num_objects":2125,"num_object_clones":588,"num_object_copies":4250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":769,"num_read":194391,"num_read_kb":871756,"num_write":18825,"num_write_kb":349504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":46033975,"num_keys_recovered":155,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15292,"ondisk_log_size":15292,"up":82,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":833120,"kb_used_data":494752,"kb_used_omap":12,"kb_used_meta":338163,"kb_avail":754141600,"statfs":{"total":773094113280,"available":772240998400,"internally_reserved":0,"allocated":506626048,"data_stored":684410932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":346279472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":89364863,"num_objects":156,"num_object_clones":64,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":105,"num_read":12843,"num_read_kb":21820,"num_write":358,"num_write_kb":6429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.510183"},"pg_stats":[{"pgid":"3.1f","version":"107'231","reported_seq":1697,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.057550+0000","last_change":"2024-09-22T01:14:06.517586+0000","last_active":"2024-09-22T01:15:48.057550+0000","last_peered":"2024-09-22T01:15:48.057550+0000","last_clean":"2024-09-22T01:15:48.057550+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:15:48.057550+0000","last_undegraded":"2024-09-22T01:15:48.057550+0000","last_fullsized":"2024-09-22T01:15:48.057550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:01.098816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051433800000000003,"stat_sum":{"num_bytes":19676866,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":1355,"num_read_kb":19829,"num_write":225,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"109'402","reported_seq":11680,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.057216+0000","last_change":"2024-09-22T01:14:38.655424+0000","last_active":"2024-09-22T01:15:48.057216+0000","last_peered":"2024-09-22T01:15:48.057216+0000","last_clean":"2024-09-22T01:15:48.057216+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:15:48.057216+0000","last_undegraded":"2024-09-22T01:15:48.057216+0000","last_fullsized":"2024-09-22T01:15:48.057216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:16.568297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049037099999999997,"stat_sum":{"num_bytes":54391157,"num_objects":64,"num_object_clones":19,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":25,"num_read":11659,"num_read_kb":35521,"num_write":461,"num_write_kb":16086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"108'291","reported_seq":14560,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051856+0000","last_change":"2024-09-22T01:14:10.568362+0000","last_active":"2024-09-22T01:15:48.051856+0000","last_peered":"2024-09-22T01:15:48.051856+0000","last_clean":"2024-09-22T01:15:48.051856+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:15:48.051856+0000","last_undegraded":"2024-09-22T01:15:48.051856+0000","last_fullsized":"2024-09-22T01:15:48.051856+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:55:35.805917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001313448,"stat_sum":{"num_bytes":49213213,"num_objects":58,"num_object_clones":16,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":16,"num_read":15310,"num_read_kb":24151,"num_write":306,"num_write_kb":10196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"108'486","reported_seq":7577,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059153+0000","last_change":"2024-09-22T01:14:06.513813+0000","last_active":"2024-09-22T01:15:48.059153+0000","last_peered":"2024-09-22T01:15:48.059153+0000","last_clean":"2024-09-22T01:15:48.059153+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:15:48.059153+0000","last_undegraded":"2024-09-22T01:15:48.059153+0000","last_fullsized":"2024-09-22T01:15:48.059153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:52:00.389015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070320599999999995,"stat_sum":{"num_bytes":48857782,"num_objects":67,"num_object_clones":19,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":23,"num_read":8249,"num_read_kb":30444,"num_write":593,"num_write_kb":12505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"109'264","reported_seq":2739,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T01:15:47.057455+0000","last_change":"2024-09-22T01:15:32.944791+0000","last_active":"2024-09-22T01:15:47.057455+0000","last_peered":"2024-09-22T01:15:47.057455+0000","last_clean":"2024-09-22T01:15:47.057455+0000","last_became_active":"2024-09-22T01:15:32.940539+0000","last_became_peered":"2024-09-22T01:15:32.940539+0000","last_unstale":"2024-09-22T01:15:47.057455+0000","last_undegraded":"2024-09-22T01:15:47.057455+0000","last_fullsized":"2024-09-22T01:15:47.057455+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:01:45.467013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001828959,"stat_sum":{"num_bytes":51317962,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":25,"num_read":2636,"num_read_kb":7589,"num_write":233,"num_write_kb":4307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'958","reported_seq":14344,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059415+0000","last_change":"2024-09-22T01:14:06.514425+0000","last_active":"2024-09-22T01:15:48.059415+0000","last_peered":"2024-09-22T01:15:48.059415+0000","last_clean":"2024-09-22T01:15:48.059415+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:15:48.059415+0000","last_undegraded":"2024-09-22T01:15:48.059415+0000","last_fullsized":"2024-09-22T01:15:48.059415+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":958,"log_dups_size":0,"ondisk_log_size":958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:35:20.174372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013489210000000001,"stat_sum":{"num_bytes":69322571,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":21,"num_read":17388,"num_read_kb":75038,"num_write":1564,"num_write_kb":26393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"108'305","reported_seq":4298,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059198+0000","last_change":"2024-09-22T01:15:32.938982+0000","last_active":"2024-09-22T01:15:48.059198+0000","last_peered":"2024-09-22T01:15:48.059198+0000","last_clean":"2024-09-22T01:15:48.059198+0000","last_became_active":"2024-09-22T01:15:32.936966+0000","last_became_peered":"2024-09-22T01:15:32.936966+0000","last_unstale":"2024-09-22T01:15:48.059198+0000","last_undegraded":"2024-09-22T01:15:48.059198+0000","last_fullsized":"2024-09-22T01:15:48.059198+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:38:09.698558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086655200000000003,"stat_sum":{"num_bytes":64137965,"num_objects":70,"num_object_clones":20,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":27,"num_read":5304,"num_read_kb":25762,"num_write":277,"num_write_kb":5531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"107'258","reported_seq":4738,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.057235+0000","last_change":"2024-09-22T01:14:06.517765+0000","last_active":"2024-09-22T01:15:48.057235+0000","last_peered":"2024-09-22T01:15:48.057235+0000","last_clean":"2024-09-22T01:15:48.057235+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:15:48.057235+0000","last_undegraded":"2024-09-22T01:15:48.057235+0000","last_fullsized":"2024-09-22T01:15:48.057235+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:00:43.398425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064395299999999995,"stat_sum":{"num_bytes":52139626,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":4590,"num_read_kb":37491,"num_write":253,"num_write_kb":17477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"108'378","reported_seq":3742,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.057825+0000","last_change":"2024-09-22T01:14:10.550475+0000","last_active":"2024-09-22T01:15:48.057825+0000","last_peered":"2024-09-22T01:15:48.057825+0000","last_clean":"2024-09-22T01:15:48.057825+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:15:48.057825+0000","last_undegraded":"2024-09-22T01:15:48.057825+0000","last_fullsized":"2024-09-22T01:15:48.057825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:16:24.976288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059658699999999999,"stat_sum":{"num_bytes":68126137,"num_objects":66,"num_object_clones":19,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":21,"num_read":4515,"num_read_kb":32320,"num_write":401,"num_write_kb":11574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"107'361","reported_seq":10954,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.054778+0000","last_change":"2024-09-22T01:14:41.577883+0000","last_active":"2024-09-22T01:15:48.054778+0000","last_peered":"2024-09-22T01:15:48.054778+0000","last_clean":"2024-09-22T01:15:48.054778+0000","last_became_active":"2024-09-22T01:14:41.576276+0000","last_became_peered":"2024-09-22T01:14:41.576276+0000","last_unstale":"2024-09-22T01:15:48.054778+0000","last_undegraded":"2024-09-22T01:15:48.054778+0000","last_fullsized":"2024-09-22T01:15:48.054778+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:02:50.146405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077227100000000005,"stat_sum":{"num_bytes":47003754,"num_objects":69,"num_object_clones":21,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":27,"num_read":10614,"num_read_kb":24986,"num_write":407,"num_write_kb":12389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4954128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"108'686","reported_seq":2216,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T01:15:47.057522+0000","last_change":"2024-09-22T01:14:06.522175+0000","last_active":"2024-09-22T01:15:47.057522+0000","last_peered":"2024-09-22T01:15:47.057522+0000","last_clean":"2024-09-22T01:15:47.057522+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:15:47.057522+0000","last_undegraded":"2024-09-22T01:15:47.057522+0000","last_fullsized":"2024-09-22T01:15:47.057522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:41:22.309163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096651400000000002,"stat_sum":{"num_bytes":22897369,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":4214,"num_read_kb":16488,"num_write":1074,"num_write_kb":3892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"107'601","reported_seq":6409,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051503+0000","last_change":"2024-09-22T01:14:41.579663+0000","last_active":"2024-09-22T01:15:48.051503+0000","last_peered":"2024-09-22T01:15:48.051503+0000","last_clean":"2024-09-22T01:15:48.051503+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:15:48.051503+0000","last_undegraded":"2024-09-22T01:15:48.051503+0000","last_fullsized":"2024-09-22T01:15:48.051503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:38.071375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011436599999999999,"stat_sum":{"num_bytes":65747428,"num_objects":69,"num_object_clones":17,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":25,"num_read":8282,"num_read_kb":28079,"num_write":816,"num_write_kb":11806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051785+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:15:48.051785+0000","last_peered":"2024-09-22T01:15:48.051785+0000","last_clean":"2024-09-22T01:15:48.051785+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:15:48.051785+0000","last_undegraded":"2024-09-22T01:15:48.051785+0000","last_fullsized":"2024-09-22T01:15:48.051785+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"106'404","reported_seq":2947,"reported_epoch":106,"state":"remapped+peering","last_fresh":"2024-09-22T01:15:44.677566+0000","last_change":"2024-09-22T01:15:48.053654+0000","last_active":"2024-09-22T01:15:48.053654+0000","last_peered":"2024-09-22T01:15:44.677566+0000","last_clean":"2024-09-22T01:15:44.677566+0000","last_became_active":"2024-09-22T01:15:32.937080+0000","last_became_peered":"2024-09-22T01:15:32.937080+0000","last_unstale":"2024-09-22T01:15:44.677566+0000","last_undegraded":"2024-09-22T01:15:44.677566+0000","last_fullsized":"2024-09-22T01:15:44.677566+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017479349999999999,"stat_sum":{"num_bytes":35342781,"num_objects":49,"num_object_clones":15,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":5152,"num_read_kb":39840,"num_write":380,"num_write_kb":22951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":1,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T01:15:47.057761+0000","last_change":"2024-09-22T01:15:32.938118+0000","last_active":"2024-09-22T01:15:47.057761+0000","last_peered":"2024-09-22T01:15:47.057761+0000","last_clean":"2024-09-22T01:15:47.057761+0000","last_became_active":"2024-09-22T01:15:32.937355+0000","last_became_peered":"2024-09-22T01:15:32.937355+0000","last_unstale":"2024-09-22T01:15:47.057761+0000","last_undegraded":"2024-09-22T01:15:47.057761+0000","last_fullsized":"2024-09-22T01:15:47.057761+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:52:32.494366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"108'236","reported_seq":2146,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.056714+0000","last_change":"2024-09-22T01:15:32.928350+0000","last_active":"2024-09-22T01:15:48.056714+0000","last_peered":"2024-09-22T01:15:48.056714+0000","last_clean":"2024-09-22T01:15:48.056714+0000","last_became_active":"2024-09-22T01:15:32.926099+0000","last_became_peered":"2024-09-22T01:15:32.926099+0000","last_unstale":"2024-09-22T01:15:48.056714+0000","last_undegraded":"2024-09-22T01:15:48.056714+0000","last_fullsized":"2024-09-22T01:15:48.056714+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:06:56.361640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014377039999999999,"stat_sum":{"num_bytes":34385559,"num_objects":63,"num_object_clones":21,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":24,"num_read":2341,"num_read_kb":24325,"num_write":223,"num_write_kb":9788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":172,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.052052+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:15:48.052052+0000","last_peered":"2024-09-22T01:15:48.052052+0000","last_clean":"2024-09-22T01:15:48.052052+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:15:48.052052+0000","last_undegraded":"2024-09-22T01:15:48.052052+0000","last_fullsized":"2024-09-22T01:15:48.052052+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"106'487","reported_seq":2343,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.056207+0000","last_change":"2024-09-22T01:14:06.512900+0000","last_active":"2024-09-22T01:15:48.056207+0000","last_peered":"2024-09-22T01:15:48.056207+0000","last_clean":"2024-09-22T01:15:48.056207+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:15:48.056207+0000","last_undegraded":"2024-09-22T01:15:48.056207+0000","last_fullsized":"2024-09-22T01:15:48.056207+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:09:44.596450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030844190000000001,"stat_sum":{"num_bytes":29917963,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":28,"num_read":4405,"num_read_kb":18549,"num_write":669,"num_write_kb":6141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":191,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.057074+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:15:48.057074+0000","last_peered":"2024-09-22T01:15:48.057074+0000","last_clean":"2024-09-22T01:15:48.057074+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:15:48.057074+0000","last_undegraded":"2024-09-22T01:15:48.057074+0000","last_fullsized":"2024-09-22T01:15:48.057074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"109'518","reported_seq":5390,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.054608+0000","last_change":"2024-09-22T01:14:41.570935+0000","last_active":"2024-09-22T01:15:48.054608+0000","last_peered":"2024-09-22T01:15:48.054608+0000","last_clean":"2024-09-22T01:15:48.054608+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:15:48.054608+0000","last_undegraded":"2024-09-22T01:15:48.054608+0000","last_fullsized":"2024-09-22T01:15:48.054608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:23.776016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054131700000000004,"stat_sum":{"num_bytes":43622971,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":24,"num_read":6403,"num_read_kb":34345,"num_write":694,"num_write_kb":10763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"108'540","reported_seq":3228,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T01:15:47.057539+0000","last_change":"2024-09-22T01:14:38.704568+0000","last_active":"2024-09-22T01:15:47.057539+0000","last_peered":"2024-09-22T01:15:47.057539+0000","last_clean":"2024-09-22T01:15:47.057539+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:15:47.057539+0000","last_undegraded":"2024-09-22T01:15:47.057539+0000","last_fullsized":"2024-09-22T01:15:47.057539+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:48:16.983872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053589800000000004,"stat_sum":{"num_bytes":36480499,"num_objects":84,"num_object_clones":25,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":28,"num_read":4054,"num_read_kb":25581,"num_write":664,"num_write_kb":10417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":176,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051670+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:15:48.051670+0000","last_peered":"2024-09-22T01:15:48.051670+0000","last_clean":"2024-09-22T01:15:48.051670+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:15:48.051670+0000","last_undegraded":"2024-09-22T01:15:48.051670+0000","last_fullsized":"2024-09-22T01:15:48.051670+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"108'354","reported_seq":2037,"reported_epoch":109,"state":"remapped+peering","last_fresh":"2024-09-22T01:15:47.061648+0000","last_change":"2024-09-22T01:15:48.057943+0000","last_active":"2024-09-22T01:15:47.061648+0000","last_peered":"2024-09-22T01:15:47.031291+0000","last_clean":"2024-09-22T01:15:47.031291+0000","last_became_active":"2024-09-22T01:14:04.527101+0000","last_became_peered":"2024-09-22T01:14:04.527101+0000","last_unstale":"2024-09-22T01:15:47.061648+0000","last_undegraded":"2024-09-22T01:15:47.061648+0000","last_fullsized":"2024-09-22T01:15:47.061648+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058451900000000003,"stat_sum":{"num_bytes":39262733,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":2665,"num_read_kb":25715,"num_write":398,"num_write_kb":8119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.068163+0000","last_change":"2024-09-22T01:15:48.063999+0000","last_active":"2024-09-22T01:15:48.068163+0000","last_peered":"2024-09-22T01:15:48.068163+0000","last_clean":"2024-09-22T01:15:48.068163+0000","last_became_active":"2024-09-22T01:15:48.063387+0000","last_became_peered":"2024-09-22T01:15:48.063387+0000","last_unstale":"2024-09-22T01:15:48.068163+0000","last_undegraded":"2024-09-22T01:15:48.068163+0000","last_fullsized":"2024-09-22T01:15:48.068163+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:03:32.378683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"109'367","reported_seq":2450,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059834+0000","last_change":"2024-09-22T01:15:32.940653+0000","last_active":"2024-09-22T01:15:48.059834+0000","last_peered":"2024-09-22T01:15:48.059834+0000","last_clean":"2024-09-22T01:15:48.059834+0000","last_became_active":"2024-09-22T01:15:32.937217+0000","last_became_peered":"2024-09-22T01:15:32.937217+0000","last_unstale":"2024-09-22T01:15:48.059834+0000","last_undegraded":"2024-09-22T01:15:48.059834+0000","last_fullsized":"2024-09-22T01:15:48.059834+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:36:36.018908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014369280000000001,"stat_sum":{"num_bytes":55121622,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":27,"num_read":3581,"num_read_kb":7914,"num_write":388,"num_write_kb":11313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051413+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:15:48.051413+0000","last_peered":"2024-09-22T01:15:48.051413+0000","last_clean":"2024-09-22T01:15:48.051413+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:15:48.051413+0000","last_undegraded":"2024-09-22T01:15:48.051413+0000","last_fullsized":"2024-09-22T01:15:48.051413+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"108'446","reported_seq":5280,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.054484+0000","last_change":"2024-09-22T01:15:32.932715+0000","last_active":"2024-09-22T01:15:48.054484+0000","last_peered":"2024-09-22T01:15:48.054484+0000","last_clean":"2024-09-22T01:15:48.054484+0000","last_became_active":"2024-09-22T01:15:32.929890+0000","last_became_peered":"2024-09-22T01:15:32.929890+0000","last_unstale":"2024-09-22T01:15:48.054484+0000","last_undegraded":"2024-09-22T01:15:48.054484+0000","last_fullsized":"2024-09-22T01:15:48.054484+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:14.936390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012260769999999999,"stat_sum":{"num_bytes":86137966,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":21,"num_read":6925,"num_read_kb":24453,"num_write":516,"num_write_kb":9639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":194,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051718+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:15:48.051718+0000","last_peered":"2024-09-22T01:15:48.051718+0000","last_clean":"2024-09-22T01:15:48.051718+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:15:48.051718+0000","last_undegraded":"2024-09-22T01:15:48.051718+0000","last_fullsized":"2024-09-22T01:15:48.051718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":343,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059284+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:15:48.059284+0000","last_peered":"2024-09-22T01:15:48.059284+0000","last_clean":"2024-09-22T01:15:48.059284+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:15:48.059284+0000","last_undegraded":"2024-09-22T01:15:48.059284+0000","last_fullsized":"2024-09-22T01:15:48.059284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"107'329","reported_seq":1693,"reported_epoch":107,"state":"remapped+peering","last_fresh":"2024-09-22T01:15:45.102703+0000","last_change":"2024-09-22T01:15:48.053375+0000","last_active":"2024-09-22T01:15:48.053375+0000","last_peered":"2024-09-22T01:15:45.102703+0000","last_clean":"2024-09-22T01:15:45.102703+0000","last_became_active":"2024-09-22T01:15:32.937085+0000","last_became_peered":"2024-09-22T01:15:32.937085+0000","last_unstale":"2024-09-22T01:15:45.102703+0000","last_undegraded":"2024-09-22T01:15:45.102703+0000","last_fullsized":"2024-09-22T01:15:45.102703+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173763,"stat_sum":{"num_bytes":62995073,"num_objects":68,"num_object_clones":24,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":25,"num_read":1964,"num_read_kb":14598,"num_write":270,"num_write_kb":3876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":2,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"108'759","reported_seq":16594,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051709+0000","last_change":"2024-09-22T01:14:10.536507+0000","last_active":"2024-09-22T01:15:48.051709+0000","last_peered":"2024-09-22T01:15:48.051709+0000","last_clean":"2024-09-22T01:15:48.051709+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:15:48.051709+0000","last_undegraded":"2024-09-22T01:15:48.051709+0000","last_fullsized":"2024-09-22T01:15:48.051709+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:08:07.366360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062985299999999999,"stat_sum":{"num_bytes":93547847,"num_objects":85,"num_object_clones":21,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":27,"num_read":21224,"num_read_kb":78254,"num_write":808,"num_write_kb":22533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"108'402","reported_seq":2596,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051569+0000","last_change":"2024-09-22T01:14:10.542822+0000","last_active":"2024-09-22T01:15:48.051569+0000","last_peered":"2024-09-22T01:15:48.051569+0000","last_clean":"2024-09-22T01:15:48.051569+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:15:48.051569+0000","last_undegraded":"2024-09-22T01:15:48.051569+0000","last_fullsized":"2024-09-22T01:15:48.051569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:18:19.906097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045127699999999998,"stat_sum":{"num_bytes":28353765,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":5190,"num_read_kb":9881,"num_write":393,"num_write_kb":4235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"108'434","reported_seq":3045,"reported_epoch":110,"state":"remapped+peering","last_fresh":"2024-09-22T01:15:48.057570+0000","last_change":"2024-09-22T01:15:48.057570+0000","last_active":"2024-09-22T01:15:47.063948+0000","last_peered":"2024-09-22T01:15:46.101219+0000","last_clean":"2024-09-22T01:15:46.101219+0000","last_became_active":"2024-09-22T01:14:04.532354+0000","last_became_peered":"2024-09-22T01:14:04.532354+0000","last_unstale":"2024-09-22T01:15:48.057570+0000","last_undegraded":"2024-09-22T01:15:48.057570+0000","last_fullsized":"2024-09-22T01:15:48.057570+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071218400000000004,"stat_sum":{"num_bytes":59544540,"num_objects":65,"num_object_clones":21,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":22,"num_read":4171,"num_read_kb":16343,"num_write":585,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"108'534","reported_seq":3056,"reported_epoch":108,"state":"peering","last_fresh":"2024-09-22T01:15:46.373764+0000","last_change":"2024-09-22T01:15:47.058474+0000","last_active":"2024-09-22T01:15:47.058474+0000","last_peered":"2024-09-22T01:15:46.373764+0000","last_clean":"2024-09-22T01:15:46.373764+0000","last_became_active":"2024-09-22T01:14:04.520033+0000","last_became_peered":"2024-09-22T01:14:04.520033+0000","last_unstale":"2024-09-22T01:15:46.373764+0000","last_undegraded":"2024-09-22T01:15:46.373764+0000","last_fullsized":"2024-09-22T01:15:46.373764+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092613299999999999,"stat_sum":{"num_bytes":36122555,"num_objects":90,"num_object_clones":32,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":37,"num_read":3511,"num_read_kb":26070,"num_write":656,"num_write_kb":9666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"109'552","reported_seq":1491,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051579+0000","last_change":"2024-09-22T01:14:10.593159+0000","last_active":"2024-09-22T01:15:48.051579+0000","last_peered":"2024-09-22T01:15:48.051579+0000","last_clean":"2024-09-22T01:15:48.051579+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:15:48.051579+0000","last_undegraded":"2024-09-22T01:15:48.051579+0000","last_fullsized":"2024-09-22T01:15:48.051579+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:27:00.125557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049513599999999997,"stat_sum":{"num_bytes":39941453,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":18,"num_read":2874,"num_read_kb":11390,"num_write":776,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"107'703","reported_seq":2792,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.054952+0000","last_change":"2024-09-22T01:14:41.579603+0000","last_active":"2024-09-22T01:15:48.054952+0000","last_peered":"2024-09-22T01:15:48.054952+0000","last_clean":"2024-09-22T01:15:48.054952+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:15:48.054952+0000","last_undegraded":"2024-09-22T01:15:48.054952+0000","last_fullsized":"2024-09-22T01:15:48.054952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:21:10.857277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000845238,"stat_sum":{"num_bytes":34534375,"num_objects":72,"num_object_clones":20,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":25,"num_read":4433,"num_read_kb":20372,"num_write":966,"num_write_kb":11080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"108'506","reported_seq":1310,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-22T01:15:47.057991+0000","last_change":"2024-09-22T01:14:06.521504+0000","last_active":"2024-09-22T01:15:47.057991+0000","last_peered":"2024-09-22T01:15:47.057991+0000","last_clean":"2024-09-22T01:15:47.057991+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:15:47.057991+0000","last_undegraded":"2024-09-22T01:15:47.057991+0000","last_fullsized":"2024-09-22T01:15:47.057991+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:50:25.455877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057186900000000002,"stat_sum":{"num_bytes":33019719,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":26,"num_read":2075,"num_read_kb":19978,"num_write":669,"num_write_kb":10295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"108'469","reported_seq":2315,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.059188+0000","last_change":"2024-09-22T01:14:06.513388+0000","last_active":"2024-09-22T01:15:48.059188+0000","last_peered":"2024-09-22T01:15:48.059188+0000","last_clean":"2024-09-22T01:15:48.059188+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:15:48.059188+0000","last_undegraded":"2024-09-22T01:15:48.059188+0000","last_fullsized":"2024-09-22T01:15:48.059188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:56.377373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045502500000000001,"stat_sum":{"num_bytes":35854533,"num_objects":70,"num_object_clones":16,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":25,"num_read":3577,"num_read_kb":14016,"num_write":564,"num_write_kb":4414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"107'747","reported_seq":3590,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051422+0000","last_change":"2024-09-22T01:14:41.582658+0000","last_active":"2024-09-22T01:15:48.051422+0000","last_peered":"2024-09-22T01:15:48.051422+0000","last_clean":"2024-09-22T01:15:48.051422+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:15:48.051422+0000","last_undegraded":"2024-09-22T01:15:48.051422+0000","last_fullsized":"2024-09-22T01:15:48.051422+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:05:18.988390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011918090000000001,"stat_sum":{"num_bytes":47660024,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":21,"num_read":6279,"num_read_kb":48950,"num_write":991,"num_write_kb":22291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"107'631","reported_seq":3111,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.056159+0000","last_change":"2024-09-22T01:14:10.569479+0000","last_active":"2024-09-22T01:15:48.056159+0000","last_peered":"2024-09-22T01:15:48.056159+0000","last_clean":"2024-09-22T01:15:48.056159+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:15:48.056159+0000","last_undegraded":"2024-09-22T01:15:48.056159+0000","last_fullsized":"2024-09-22T01:15:48.056159+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:20:59.408962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805440000000001,"stat_sum":{"num_bytes":57588570,"num_objects":68,"num_object_clones":19,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":27,"num_read":4836,"num_read_kb":24367,"num_write":821,"num_write_kb":10550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"107'533","reported_seq":2737,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-22T01:15:48.051622+0000","last_change":"2024-09-22T01:14:10.541253+0000","last_active":"2024-09-22T01:15:48.051622+0000","last_peered":"2024-09-22T01:15:48.051622+0000","last_clean":"2024-09-22T01:15:48.051622+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:15:48.051622+0000","last_undegraded":"2024-09-22T01:15:48.051622+0000","last_fullsized":"2024-09-22T01:15:48.051622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:43:52.815444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036484199999999999,"stat_sum":{"num_bytes":59021655,"num_objects":78,"num_object_clones":21,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":32,"num_read":4499,"num_read_kb":29017,"num_write":623,"num_write_kb":6140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1561288033,"num_objects":2121,"num_object_clones":588,"num_object_copies":4242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2121,"num_whiteouts":769,"num_read":194275,"num_read_kb":871656,"num_write":18664,"num_write_kb":347086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":516,"num_bytes_recovered":46033956,"num_keys_recovered":155,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":495075328,"data_stored":678410922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15172,"ondisk_log_size":15172,"up":64,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82416,"kb_used_data":47060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94289424,"statfs":{"total":96636764160,"available":96552370176,"internally_reserved":0,"allocated":48189440,"data_stored":61401051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88212,"kb_used_data":44544,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94283628,"statfs":{"total":96636764160,"available":96546435072,"internally_reserved":0,"allocated":45613056,"data_stored":56146612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96792,"kb_used_data":69760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94275048,"statfs":{"total":96636764160,"available":96537649152,"internally_reserved":0,"allocated":71434240,"data_stored":93417089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":109512,"kb_used_data":70004,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94262328,"statfs":{"total":96636764160,"available":96524623872,"internally_reserved":0,"allocated":71684096,"data_stored":102170901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":124812,"kb_used_data":75572,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94247028,"statfs":{"total":96636764160,"available":96508956672,"internally_reserved":0,"allocated":77385728,"data_stored":107939223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":120980,"kb_used_data":71748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94250860,"statfs":{"total":96636764160,"available":96512880640,"internally_reserved":0,"allocated":73469952,"data_stored":99365071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":106360,"kb_used_data":61276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94265480,"statfs":{"total":96636764160,"available":96527851520,"internally_reserved":0,"allocated":62746624,"data_stored":87132246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104036,"kb_used_data":54788,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94267804,"statfs":{"total":96636764160,"available":96530231296,"internally_reserved":0,"allocated":56102912,"data_stored":76838739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54816768,"data_stored":76240423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60862464,"data_stored":85941514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72179712,"data_stored":98764707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":76091392,"data_stored":107338840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":70381568,"data_stored":101564413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":69537792,"data_stored":92220233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44306432,"data_stored":55540105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":46899200,"data_stored":60800687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:15:52.552 INFO:teuthology.orchestra.run.smithi096.stdout:5801 trunc from 0xe67e7cf to 0xa1d8edf 2024-09-22T01:15:52.562 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:52.562 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:52.571 INFO:teuthology.orchestra.run.smithi096.stdout:5803 read 0x8f85679 thru 0x8f92595 (0xcf1d bytes) 2024-09-22T01:15:52.572 INFO:teuthology.orchestra.run.smithi096.stdout:5804 writesame 0x8941ae1 thru 0x8945917 data_size 0x1(0x3e37 bytes) 2024-09-22T01:15:52.585 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:52.585 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:52.592 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.592 INFO:teuthology.orchestra.run.smithi096.stdout:5806 writesame 0x9bff36f thru 0x9c05bb6 data_size 0x2f(0x6848 bytes) 2024-09-22T01:15:52.593 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:52.594 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:52.597 INFO:teuthology.orchestra.run.smithi096.stdout:5807 read 0xbf205 thru 0xc9804 (0xa600 bytes) 2024-09-22T01:15:52.604 INFO:teuthology.orchestra.run.smithi096.stdout:5808 trunc from 0xa1d8edf to 0x5ba2fcc 2024-09-22T01:15:52.611 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:52.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:52.643 INFO:teuthology.orchestra.run.smithi096.stdout:5809 write 0x2cc9c1e thru 0x2ccb5a0 (0x1983 bytes) 2024-09-22T01:15:52.644 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:52.640+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:52.651 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.669 INFO:teuthology.orchestra.run.smithi096.stdout:5812 writesame 0xe88b67a thru 0xe88ee0a data_size 0x5(0x3791 bytes) 2024-09-22T01:15:52.669 INFO:teuthology.orchestra.run.smithi096.stdout:5813 write 0x3c98e4a thru 0x3ca69a9 (0xdb60 bytes) 2024-09-22T01:15:52.691 INFO:teuthology.orchestra.run.smithi096.stdout:5816 read 0xa04efe6 thru 0xa057059 (0x8074 bytes) 2024-09-22T01:15:52.691 INFO:teuthology.orchestra.run.smithi096.stdout:5818 read 0xba8b465 thru 0xba94d43 (0x98df bytes) 2024-09-22T01:15:52.691 INFO:teuthology.orchestra.run.smithi096.stdout:5819 write 0x98ccb9f thru 0x98d195c (0x4dbe bytes) 2024-09-22T01:15:52.710 INFO:teuthology.orchestra.run.smithi096.stdout:5820 punch from 0x8f0de89 to 0x8f1d174, (0xf2eb bytes) 2024-09-22T01:15:52.710 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.712 INFO:teuthology.orchestra.run.smithi096.stdout:5822 trunc from 0xe88ee0b to 0xc674f47 2024-09-22T01:15:52.719 INFO:teuthology.orchestra.run.smithi096.stdout:5823 write 0x2760d46 thru 0x276d5b0 (0xc86b bytes) 2024-09-22T01:15:52.738 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:52.736+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:52.740 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.740 INFO:teuthology.orchestra.run.smithi096.stdout:5826 punch from 0x61f31e7 to 0x61f7b76, (0x498f bytes) 2024-09-22T01:15:52.740 INFO:teuthology.orchestra.run.smithi096.stdout:5827 read 0x5f95039 thru 0x5fa2310 (0xd2d8 bytes) 2024-09-22T01:15:52.741 INFO:teuthology.orchestra.run.smithi096.stdout:5828 writesame 0xaa5cb2d thru 0xaa619db data_size 0x1(0x4eaf bytes) 2024-09-22T01:15:52.761 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.761 INFO:teuthology.orchestra.run.smithi096.stdout:5830 write 0xc435fcb thru 0xc43ef5d (0x8f93 bytes) 2024-09-22T01:15:52.778 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:52.778 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:52.780 INFO:teuthology.orchestra.run.smithi096.stdout:5831 read 0x446090f thru 0x4460a97 (0x189 bytes) 2024-09-22T01:15:52.780 INFO:teuthology.orchestra.run.smithi096.stdout:5833 punch from 0x4b2df4c to 0x4b36f6b, (0x901f bytes) 2024-09-22T01:15:52.780 INFO:teuthology.orchestra.run.smithi096.stdout:5834 read 0x5f50141 thru 0x5f53750 (0x3610 bytes) 2024-09-22T01:15:52.780 INFO:teuthology.orchestra.run.smithi096.stdout:5835 read 0x46325c2 thru 0x463b708 (0x9147 bytes) 2024-09-22T01:15:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:5838 read 0x710aa1 thru 0x71d651 (0xcbb1 bytes) 2024-09-22T01:15:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:5839 punch from 0x46d9460 to 0x46e1c9d, (0x883d bytes) 2024-09-22T01:15:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:5841 write 0x1b037d2 thru 0x1b0b0d2 (0x7901 bytes) 2024-09-22T01:15:52.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:52.801 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.802 INFO:teuthology.orchestra.run.smithi096.stdout:5844 writesame 0x9cd7c48 thru 0x9cd9998 data_size 0x13(0x1d51 bytes) 2024-09-22T01:15:52.812 INFO:teuthology.orchestra.run.smithi096.stdout:5846 read 0x963ed7 thru 0x967760 (0x388a bytes) 2024-09-22T01:15:52.815 INFO:teuthology.orchestra.run.smithi096.stdout:5847 trunc from 0xc674f47 to 0xda414dc 2024-09-22T01:15:52.821 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.821 INFO:teuthology.orchestra.run.smithi096.stdout:5851 write 0x5d9bd57 thru 0x5d9eb68 (0x2e12 bytes) 2024-09-22T01:15:52.838 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.839 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:52.836+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:52.841 INFO:teuthology.orchestra.run.smithi096.stdout:5853 trunc from 0xda414dc to 0xba13345 2024-09-22T01:15:52.858 INFO:teuthology.orchestra.run.smithi096.stdout:5854 read 0x4f88290 thru 0x4f95c9c (0xda0d bytes) 2024-09-22T01:15:52.858 INFO:teuthology.orchestra.run.smithi096.stdout:5855 writesame 0x680845b thru 0x6817dc6 data_size 0x66(0xf96c bytes) 2024-09-22T01:15:52.879 INFO:teuthology.orchestra.run.smithi096.stdout:5856 trunc from 0xba13345 to 0xd162ce9 2024-09-22T01:15:52.899 INFO:teuthology.orchestra.run.smithi096.stdout:5860 trunc from 0xd162ce9 to 0x16b9e10 2024-09-22T01:15:52.936 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:52.937 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:52.940 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:52.936+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:52.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:52.953 INFO:teuthology.orchestra.run.smithi096.stdout:5861 trunc from 0x16b9e10 to 0xce8fc7 2024-09-22T01:15:52.971 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.971 INFO:teuthology.orchestra.run.smithi096.stdout:5863 read 0x8ab041 thru 0x8acac3 (0x1a83 bytes) 2024-09-22T01:15:52.971 INFO:teuthology.orchestra.run.smithi096.stdout:5864 read 0x1a8f30 thru 0x1aa099 (0x116a bytes) 2024-09-22T01:15:52.972 INFO:teuthology.orchestra.run.smithi096.stdout:5865 punch from 0x2cf7dc to 0x2d207b, (0x289f bytes) 2024-09-22T01:15:52.972 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.973 INFO:teuthology.orchestra.run.smithi096.stdout:5867 read 0xa50c5b thru 0xa522ca (0x1670 bytes) 2024-09-22T01:15:52.986 INFO:teuthology.orchestra.run.smithi096.stdout:5868 writesame 0x63d2336 thru 0x63db74b data_size 0x55(0x9416 bytes) 2024-09-22T01:15:52.986 INFO:teuthology.orchestra.run.smithi096.stdout:5869 punch from 0x1ef70d2 to 0x1f004b4, (0x93e2 bytes) 2024-09-22T01:15:52.987 INFO:teuthology.orchestra.run.smithi096.stdout:5870 write 0x2fcbec8 thru 0x2fcc297 (0x3d0 bytes) 2024-09-22T01:15:52.992 INFO:teuthology.orchestra.run.smithi096.stdout:5872 punch from 0x254d753 to 0x255893d, (0xb1ea bytes) 2024-09-22T01:15:52.992 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:52.992 INFO:teuthology.orchestra.run.smithi096.stdout:5877 write 0x4f35d thru 0x517d0 (0x2474 bytes) 2024-09-22T01:15:53.010 INFO:teuthology.orchestra.run.smithi096.stdout:5878 writesame 0xc7c2017 thru 0xc7c659d data_size 0x11(0x4587 bytes) 2024-09-22T01:15:53.011 INFO:teuthology.orchestra.run.smithi096.stdout:5879 read 0x64dcc7d thru 0x64e7616 (0xa99a bytes) 2024-09-22T01:15:53.017 INFO:teuthology.orchestra.run.smithi096.stdout:5880 write 0xcc96f90 thru 0xcc9f044 (0x80b5 bytes) 2024-09-22T01:15:53.041 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:53.036+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:53.052 INFO:teuthology.orchestra.run.smithi096.stdout:5881 trunc from 0xcc9f045 to 0x26185ab 2024-09-22T01:15:53.082 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.082 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.086 INFO:teuthology.orchestra.run.smithi096.stdout:5882 write 0x8a728cc thru 0x8a79281 (0x69b6 bytes) 2024-09-22T01:15:53.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:53.109 INFO:teuthology.orchestra.run.smithi096.stdout:5883 write 0x6d57acb thru 0x6d62d42 (0xb278 bytes) 2024-09-22T01:15:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:5884 read 0x47a010e thru 0x47a511e (0x5011 bytes) 2024-09-22T01:15:53.132 INFO:teuthology.orchestra.run.smithi096.stdout:5885 read 0x14e684 thru 0x1514a7 (0x2e24 bytes) 2024-09-22T01:15:53.134 INFO:teuthology.orchestra.run.smithi096.stdout:5886 trunc from 0x8a79282 to 0x7c44a2c 2024-09-22T01:15:53.141 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:53.140+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:53.168 INFO:teuthology.orchestra.run.smithi096.stdout:5887 writesame 0xbb9ed7d thru 0xbbab848 data_size 0x2(0xcacc bytes) 2024-09-22T01:15:53.169 INFO:teuthology.orchestra.run.smithi096.stdout:5888 punch from 0x2e247c6 to 0x2e2959e, (0x4dd8 bytes) 2024-09-22T01:15:53.169 INFO:teuthology.orchestra.run.smithi096.stdout:5889 write 0x2fafabb thru 0x2fb8f5b (0x94a1 bytes) 2024-09-22T01:15:53.190 INFO:teuthology.orchestra.run.smithi096.stdout:5890 writesame 0x4b95fdd thru 0x4b9d100 data_size 0x1a(0x7124 bytes) 2024-09-22T01:15:53.216 INFO:teuthology.orchestra.run.smithi096.stdout:5891 read 0x86092c1 thru 0x861731e (0xe05e bytes) 2024-09-22T01:15:53.216 INFO:teuthology.orchestra.run.smithi096.stdout:5892 write 0x8dd71be thru 0x8de33ec (0xc22f bytes) 2024-09-22T01:15:53.238 INFO:teuthology.orchestra.run.smithi096.stdout:5893 punch from 0x30db650 to 0x30e79ba, (0xc36a bytes) 2024-09-22T01:15:53.242 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:53.236+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:53.244 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.244 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.245 INFO:teuthology.orchestra.run.smithi096.stdout:5894 trunc from 0xbbab849 to 0x5df3018 2024-09-22T01:15:53.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.270 INFO:teuthology.orchestra.run.smithi096.stdout:5896 writesame 0x553da95 thru 0x5542bf1 data_size 0x3(0x515d bytes) 2024-09-22T01:15:53.294 INFO:teuthology.orchestra.run.smithi096.stdout:5897 read 0x2a2b2f6 thru 0x2a2b92d (0x638 bytes) 2024-09-22T01:15:53.311 INFO:teuthology.orchestra.run.smithi096.stdout:5898 writesame 0xe0c7f3d thru 0xe0c86a9 data_size 0x1(0x76d bytes) 2024-09-22T01:15:53.311 INFO:teuthology.orchestra.run.smithi096.stdout:5899 read 0x16390ae thru 0x1640836 (0x7789 bytes) 2024-09-22T01:15:53.312 INFO:teuthology.orchestra.run.smithi096.stdout:5900 read 0x1a24c63 thru 0x1a24edb (0x279 bytes) 2024-09-22T01:15:53.312 INFO:teuthology.orchestra.run.smithi096.stdout:5901 punch from 0xba6a646 to 0xba72a98, (0x8452 bytes) 2024-09-22T01:15:53.312 INFO:teuthology.orchestra.run.smithi096.stdout:5902 write 0x13e5078 thru 0x13edb23 (0x8aac bytes) 2024-09-22T01:15:53.333 INFO:teuthology.orchestra.run.smithi096.stdout:5903 write 0x86cd7bf thru 0x86d1470 (0x3cb2 bytes) 2024-09-22T01:15:53.343 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:53.340+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:53.351 INFO:teuthology.orchestra.run.smithi096.stdout:5904 write 0x14a770e thru 0x14af74f (0x8042 bytes) 2024-09-22T01:15:53.359 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.359 INFO:teuthology.orchestra.run.smithi096.stdout:5906 read 0xaac623a thru 0xaacec73 (0x8a3a bytes) 2024-09-22T01:15:53.367 INFO:teuthology.orchestra.run.smithi096.stdout:5907 write 0xeaa4bfb thru 0xeab366e (0xea74 bytes) 2024-09-22T01:15:53.389 INFO:teuthology.orchestra.run.smithi096.stdout:5908 read 0x14dbe2a thru 0x14e1b39 (0x5d10 bytes) 2024-09-22T01:15:53.390 INFO:teuthology.orchestra.run.smithi096.stdout:5909 writesame 0x11c050f thru 0x11c57ab data_size 0x1(0x529d bytes) 2024-09-22T01:15:53.403 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.403 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.412 INFO:teuthology.orchestra.run.smithi096.stdout:5912 write 0x87bd547 thru 0x87c919d (0xbc57 bytes) 2024-09-22T01:15:53.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:53.428 INFO:teuthology.orchestra.run.smithi096.stdout:5915 write 0xcea084 thru 0xcf95eb (0xf568 bytes) 2024-09-22T01:15:53.451 INFO:teuthology.orchestra.run.smithi096.stdout:5916 read 0x767e70f thru 0x767f8a3 (0x1195 bytes) 2024-09-22T01:15:53.451 INFO:teuthology.orchestra.run.smithi096.stdout:5917 punch from 0x431635e to 0x4323158, (0xcdfa bytes) 2024-09-22T01:15:53.452 INFO:teuthology.orchestra.run.smithi096.stdout:5918 write 0x67605dc thru 0x676d52c (0xcf51 bytes) 2024-09-22T01:15:53.452 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:53.448+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:53.473 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.473 INFO:teuthology.orchestra.run.smithi096.stdout:5920 write 0xbd43d4d thru 0xbd4a1c2 (0x6476 bytes) 2024-09-22T01:15:53.494 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.494 INFO:teuthology.orchestra.run.smithi096.stdout:5926 writesame 0x6788f18 thru 0x678bbd8 data_size 0x39(0x2cc1 bytes) 2024-09-22T01:15:53.499 INFO:teuthology.orchestra.run.smithi096.stdout:5927 write 0x537b663 thru 0x537b869 (0x207 bytes) 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:5928 punch from 0xcc11384 to 0xcc20883, (0xf4ff bytes) 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:5934 punch from 0xd55dc82 to 0xd56cdb5, (0xf133 bytes) 2024-09-22T01:15:53.503 INFO:teuthology.orchestra.run.smithi096.stdout:5937 write 0xb6bab3a thru 0xb6c90f2 (0xe5b9 bytes) 2024-09-22T01:15:53.525 INFO:teuthology.orchestra.run.smithi096.stdout:5940 read 0x5487860 thru 0x54939d4 (0xc175 bytes) 2024-09-22T01:15:53.526 INFO:teuthology.orchestra.run.smithi096.stdout:5941 write 0xb03c187 thru 0xb03fe1e (0x3c98 bytes) 2024-09-22T01:15:53.544 INFO:teuthology.orchestra.run.smithi096.stdout:5942 read 0x190dbd thru 0x195031 (0x4275 bytes) 2024-09-22T01:15:53.544 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:53.544+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:53.546 INFO:teuthology.orchestra.run.smithi096.stdout:5943 read 0xc0ca61e thru 0xc0cef5e (0x4941 bytes) 2024-09-22T01:15:53.547 INFO:teuthology.orchestra.run.smithi096.stdout:5945 write 0x1930d0c thru 0x1932335 (0x162a bytes) 2024-09-22T01:15:53.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:15:53.556 INFO:teuthology.orchestra.run.smithi096.stdout:5946 read 0xe746d90 thru 0xe748593 (0x1804 bytes) 2024-09-22T01:15:53.556 INFO:teuthology.orchestra.run.smithi096.stdout:5947 write 0xae16768 thru 0xae21dad (0xb646 bytes) 2024-09-22T01:15:53.568 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.568 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.575 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.576 INFO:teuthology.orchestra.run.smithi096.stdout:5949 write 0x8eca62 thru 0x8f6b2b (0xa0ca bytes) 2024-09-22T01:15:53.584 INFO:teuthology.orchestra.run.smithi096.stdout:5951 write 0x6edb2fc thru 0x6ee9fc2 (0xecc7 bytes) 2024-09-22T01:15:53.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:53.606 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.606 INFO:teuthology.orchestra.run.smithi096.stdout:5957 punch from 0x25da590 to 0x25e4b91, (0xa601 bytes) 2024-09-22T01:15:53.606 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.610 INFO:teuthology.orchestra.run.smithi096.stdout:5959 trunc from 0xeab366f to 0xc20da8f 2024-09-22T01:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout:5960 read 0x98bf88a thru 0x98c0b1f (0x1296 bytes) 2024-09-22T01:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:53.629 INFO:teuthology.orchestra.run.smithi096.stdout:5962 writesame 0xdb2ed2 thru 0xdbded1 data_size 0x80(0xb000 bytes) 2024-09-22T01:15:53.637 INFO:teuthology.orchestra.run.smithi096.stdout:5963 trunc from 0xc20da8f to 0xaf42867 2024-09-22T01:15:53.645 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:53.644+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:53.681 INFO:teuthology.orchestra.run.smithi096.stdout:5964 trunc from 0xaf42867 to 0x9dc812c 2024-09-22T01:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout:5965 read 0x4444ecd thru 0x444da6c (0x8ba0 bytes) 2024-09-22T01:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout:5966 punch from 0x7eb5303 to 0x7ec1b3f, (0xc83c bytes) 2024-09-22T01:15:53.704 INFO:teuthology.orchestra.run.smithi096.stdout:5967 clone 51 order 18 su 4096 sc 13 2024-09-22T01:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.734 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:53.749 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:53.748+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:53.849 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:53.844+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:53.904 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:53.905 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:53.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:53.950 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:53.948+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:54.046 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.046 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.051 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:54.048+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:54.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:54.152 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:54.148+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:54.185 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.186 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:54.252 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:54.252+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:54.353 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:54.348+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:54.365 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.366 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.380 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:54.453 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:54.452+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:54.549 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.550 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.554 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:54.552+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:54.559 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:15:54.565 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:54.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:54.661 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone50 from 0x9dc812c (overlap 0x38e88f) to 0x0 2024-09-22T01:15:54.662 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:54.660+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:54.698 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.698 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.715 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:54.766 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:54.764+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:54.800 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:54.800 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-22T01:15:54.816 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:54.817 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:54.833 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:54.833 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:54.843 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:54.843 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:54.849 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:54.849 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:54.857 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:54.856+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:54.858 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:54.875 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:54.876 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:54.906 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:54.906 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:54.939 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:54.939 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:54.958 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:54.952+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:54.962 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:54.962 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:55.023 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:55.024 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:55.058 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:55.056+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:55.159 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:55.156+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:55.200 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.200 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:55.200 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:55.200 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:55.200 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.214 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:55.259 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:55.256+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:55.361 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:55.356+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:55.387 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.387 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:55.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:55.387 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.167723504, 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:15:55.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:15:55.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:15:55.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1676934239999999, 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:15:55.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:15:55.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.167672193, 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:15:55.396 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:15:55.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:15:55.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.400 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1676509290000001, 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:55.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:15:55.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:15:55.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1676296220000002, 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:55.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:15:55.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:15:55.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1676079819999998, 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.409 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:15:55.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:15:55.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.412 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1675628520000001, 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:15:55.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1675318269999999, 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.417 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:55.421 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1675077890000001, 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1674865250000002, 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.167434493, 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:15:55.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:55.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:15:55.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1673755219999999, 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1672717549999998, 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:55.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:15:55.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:55.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1666919729999998, 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:15:55.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:15:55.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1665715300000001, 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.450 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1664915300000001, 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:15:55.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:15:55.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:15:55.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:15:55.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.166027342, 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1659577670000001, 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1658910179999999, 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:15:55.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:15:55.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 3.1658194000000002, 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:15:55.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.471 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.472 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:15:55.473 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:55.472+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:55.573 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:55.572+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:55.643 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.643 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:55.643 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:15:55.643 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:15:55.674 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:55.672+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:55.775 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:55.772+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:55.781 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.782 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:55.782 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:55.782 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:55.782 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:15:55.875 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:55.872+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:55.916 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:55.916 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:55.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:55.916 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:55.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4069481340000003, 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.920 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4068895040000005, 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.921 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.922 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:15:55.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4068666239999992, 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:15:55.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.928 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4068485610000003, 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.931 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4068110740000002, 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.932 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4067854519999994, 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:15:55.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4067646039999993, 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:15:55.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4067473649999993, 2024-09-22T01:15:55.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:55.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4066952710000002, 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:55.951 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4066753980000009, 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.955 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4066291500000005, 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4066084629999995, 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.959 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4065597739999998, 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:15:55.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4064907210000008, 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.966 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:15:55.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:15:55.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4064708869999993, 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.970 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4064222560000008, 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:15:55.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4064011960000009, 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.977 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.980 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4063506330000006, 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4063297059999993, 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 9.4063018189999994, 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:15:55.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:15:55.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:15:55.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.991 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:55.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:55.992 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:55.993 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:55.988+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:55.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:15:56.094 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:56.088+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:56.144 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:56.144 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:56.144 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:56.144 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:56.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:15:56.195 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:56.192+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:56.284 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:56.284 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:56.285 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:56.285 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:56.285 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:56.295 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:56.292+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:56.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:15:56.398 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:56.396+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:56.414 INFO:teuthology.orchestra.run.smithi096.stdout:5968 write 0xecbc04b thru 0xecc7e85 (0xbe3b bytes) 2024-09-22T01:15:56.426 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.481511935, 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:15:56.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:15:56.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:15:56.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.433 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.325662634, 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:15:56.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.325641305000001, 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:15:56.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:15:56.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:56.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.325178231999999, 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:56.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:15:56.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:15:56.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.325151362, 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:15:56.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.450 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.325127854000002, 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:56.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:15:56.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:56.454 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.324930907999999, 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:15:56.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.458 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.324902842, 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:15:56.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:15:56.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:15:56.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323661402999999, 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:15:56.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323623637000001, 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323599683000001, 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:15:56.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:15:56.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:56.475 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323578597000001, 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:15:56.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323558132999999, 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.484 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323538112000001, 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323518914000001, 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323497801999999, 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.32347721, 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:15:56.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323455848999998, 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:15:56.502 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323434535000001, 2024-09-22T01:15:56.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:15:56.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:15:56.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.323415517000001, 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:56.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:56.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:15:56.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:56.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:56.515 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:56.515 INFO:teuthology.orchestra.run.smithi096.stdout:5969 write 0xda6d1d thru 0xda7a4f (0xd33 bytes) 2024-09-22T01:15:56.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:15:56.517 INFO:teuthology.orchestra.run.smithi096.stdout:5970 read 0x2ef8424 thru 0x2f0274a (0xa327 bytes) 2024-09-22T01:15:56.517 INFO:teuthology.orchestra.run.smithi096.stdout:5971 read 0x14fc4a0 thru 0x150bc37 (0xf798 bytes) 2024-09-22T01:15:56.517 INFO:teuthology.orchestra.run.smithi096.stdout:5973 clone 52 order 22 su 32768 sc 6 2024-09-22T01:15:56.517 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:56.512+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:56.585 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:15:56.617 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:56.612+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:56.694 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:15:56.694 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:56.700 INFO:teuthology.orchestra.run.smithi096.stdout:truncating image image_client.0-clone51 from 0xecc7e86 (overlap 0x9dc812c) to 0x5eab3e7 2024-09-22T01:15:56.717 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:56.712+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:56.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:15:56.820 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:56.816+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:56.925 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:56.920+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:56.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:15:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:15:56.969 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:15:56.969 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:56.978 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:15:56.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:15:57.036 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:57.036+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:57.121 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:57.116+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:57.222 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:57.220+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:57.226 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:57.226 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.25325456500001, 2024-09-22T01:15:57.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:15:57.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047506905999995, 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047474133999998, 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:15:57.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047452669999998, 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:57.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047431727000003, 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:15:57.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:15:57.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047411819000004, 2024-09-22T01:15:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:15:57.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047391438000005, 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.268 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:15:57.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.047369501000006, 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:15:57.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:15:57.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.046859920000003, 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:15:57.276 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:15:57.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.315478873000004, 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:15:57.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:15:57.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:15:57.284 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.315444569, 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.285 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:15:57.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:15:57.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.315373925000003, 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:15:57.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:15:57.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:15:57.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.315222220999999, 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:15:57.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:15:57.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:15:57.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.315006683999997, 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:15:57.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:15:57.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.314793459000001, 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:15:57.302 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.306 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.314595335, 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:15:57.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.314391542999999, 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.314140717999997, 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:15:57.319 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.313921698000001, 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:57.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:15:57.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.6268840899999999, 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:15:57.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:15:57.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:15:57.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:15:57.330 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:15:57.331 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:57.331 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:57.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:15:57.332 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:57.333 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:57.333 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:57.333 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:57.334 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:57.334 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:57.334 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:57.328+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:57.335 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:57.335 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:57.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:57.356 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:57.432 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:57.428+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:57.485 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:57.485 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:57.504 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:15:57.543 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:15:57.543 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:15:57.543 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-22T01:15:57.543 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.418540 2024-09-22T01:15:57.543 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.41853963231227653 2024-09-22T01:15:57.548 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:57.548+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:57.583 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:15:57.642 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:57.642 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:57.642 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:57.640+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:57.656 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:15:57.734 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:57.732+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:57.763 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-22T01:15:57.765 INFO:teuthology.orchestra.run.smithi096.stdout:5974 write 0x1baf0bf thru 0x1bbb14b (0xc08d bytes) 2024-09-22T01:15:57.835 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:57.832+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:57.859 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:57.859 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:57.878 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:15:57.887 INFO:teuthology.orchestra.run.smithi096.stdout:5975 punch from 0xd27f1f8 to 0xd289e78, (0xac80 bytes) 2024-09-22T01:15:57.887 INFO:teuthology.orchestra.run.smithi096.stdout:5976 write 0xbea67a6 thru 0xbeb067e (0x9ed9 bytes) 2024-09-22T01:15:57.899 INFO:teuthology.orchestra.run.smithi096.stdout:5977 writesame 0x8366b37 thru 0x836729e data_size 0x8(0x768 bytes) 2024-09-22T01:15:57.935 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:57.932+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:57.949 INFO:teuthology.orchestra.run.smithi096.stdout:5978 trunc from 0xecc7e86 to 0x3d5f581 2024-09-22T01:15:58.026 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.026 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.037 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:58.032+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:58.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:15:58.067 INFO:teuthology.orchestra.run.smithi096.stdout:5979 writesame 0x63af23f thru 0x63b42f6 data_size 0x7b(0x50b8 bytes) 2024-09-22T01:15:58.071 INFO:teuthology.orchestra.run.smithi096.stdout:5980 trunc from 0x63b42f7 to 0x9bbc4ab 2024-09-22T01:15:58.079 INFO:teuthology.orchestra.run.smithi096.stdout:5982 read 0x2b015f0 thru 0x2b0a4f3 (0x8f04 bytes) 2024-09-22T01:15:58.080 INFO:teuthology.orchestra.run.smithi096.stdout:5983 read 0x78359c8 thru 0x784064c (0xac85 bytes) 2024-09-22T01:15:58.080 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:58.080 INFO:teuthology.orchestra.run.smithi096.stdout:5986 writesame 0x9540aba thru 0x954321f data_size 0x29(0x2766 bytes) 2024-09-22T01:15:58.087 INFO:teuthology.orchestra.run.smithi096.stdout:5987 write 0x64839c4 thru 0x6485345 (0x1982 bytes) 2024-09-22T01:15:58.093 INFO:teuthology.orchestra.run.smithi096.stdout:compare and write disabled 2024-09-22T01:15:58.105 INFO:teuthology.orchestra.run.smithi096.stdout:5990 writesame 0xbea5ed1 thru 0xbeb1bc8 data_size 0x4(0xbcf8 bytes) 2024-09-22T01:15:58.105 INFO:teuthology.orchestra.run.smithi096.stdout:5991 write 0xb2df8ff thru 0xb2ebd46 (0xc448 bytes) 2024-09-22T01:15:58.114 INFO:teuthology.orchestra.run.smithi096.stdout:5992 read 0xb05a872 thru 0xb069403 (0xeb92 bytes) 2024-09-22T01:15:58.114 INFO:teuthology.orchestra.run.smithi096.stdout:5993 write 0x4dd61a6 thru 0x4de0ad4 (0xa92f bytes) 2024-09-22T01:15:58.121 INFO:teuthology.orchestra.run.smithi096.stdout:5994 write 0xc37533 thru 0xc3d4ef (0x5fbd bytes) 2024-09-22T01:15:58.139 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:58.136+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:58.221 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.221 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.245 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:58.236+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:58.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:15:58.339 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:15:58.336+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:15:58.400 INFO:teuthology.orchestra.run.smithi096.stdout:5995 read 0x2946d45 thru 0x2953746 (0xca02 bytes) 2024-09-22T01:15:58.401 INFO:teuthology.orchestra.run.smithi096.stdout:5996 read 0x795b72f thru 0x7964112 (0x89e4 bytes) 2024-09-22T01:15:58.401 INFO:teuthology.orchestra.run.smithi096.stdout:5997 write 0x735b096 thru 0x73636a7 (0x8612 bytes) 2024-09-22T01:15:58.410 INFO:teuthology.orchestra.run.smithi096.stdout:5998 punch from 0x3ab8849 to 0x3ac2921, (0xa0d8 bytes) 2024-09-22T01:15:58.410 INFO:teuthology.orchestra.run.smithi096.stdout:5999 read 0x5844d9d thru 0x5854bfc (0xfe60 bytes) 2024-09-22T01:15:58.419 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.419 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.434 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:15:58.439 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:58.436+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:58.478 INFO:teuthology.orchestra.run.smithi096.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-09-22T01:15:58.540 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:58.540+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:58.574 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.574 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.589 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.4 primary-affinity to 0.41854 (8274292) 2024-09-22T01:15:58.591 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:15:58.673 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:58.672+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:58.722 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.722 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:15:58.745 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:58.740+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:58.844 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:58.840+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:58.886 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:58.886 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:58.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:15:58.945 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:58.944+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:59.038 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:59.038 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:59.047 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:59.044+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:59.052 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:15:59.147 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:59.144+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:59.196 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:59.196 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:59.211 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:15:59.248 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:15:59.248+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:15:59.345 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:59.345 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:59.349 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:15:59.344+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:15:59.362 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:15:59.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:15:59.449 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:15:59.444+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:15:59.503 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:15:59.504 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:15:59.518 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:15:59.519 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:15:59.523 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:15:59.523 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:15:59.534 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:15:59.534 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:15:59.537 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:15:59.549 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:15:59.550 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:15:59.550 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:15:59.548+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:15:59.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:15:59.642 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:15:59.642 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:15:59.651 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:15:59.648+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:15:59.674 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:15:59.675 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:15:59.720 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:15:59.720 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:15:59.752 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:15:59.747+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:15:59.774 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:15:59.774 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:15:59.775 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:15:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:15:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:15:59.776 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:15:59.799 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:15:59.854 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:15:59.848+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:59.938 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7183257159999998, 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:59.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:15:59.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:15:59.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7182956359999997, 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.943 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:15:59.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:15:59.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7182744049999998, 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:15:59.947 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.949 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:15:59.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:15:59.951 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7182531409999999, 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.952 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:15:59.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.718231834, 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:15:59.956 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:15:59.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7182101940000001, 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:15:59.960 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.962 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.964 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7181650639999999, 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:15:59.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.968 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7181340389999997, 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:15:59.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7181100010000003, 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.973 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7180887370000004, 2024-09-22T01:15:59.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:15:59.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:15:59.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.981 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7180367050000003, 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:59.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:15:59.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:15:59.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.985 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7179777339999998, 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:15:59.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:15:59.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:59.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7178739670000001, 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7172941850000001, 2024-09-22T01:15:59.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:15:59.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7171737419999999, 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:59.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:15:59.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7170937420000003, 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7166295539999998, 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7165599790000003, 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:00.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7164932300000002, 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:00.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:00.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 7.7164216119999995, 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:00.018 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.022 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:00.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:00.024 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:00.020+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:00.124 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:00.123+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:00.201 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:00.201 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:00.201 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:00.201 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:00.220 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:00.263 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:00.260+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:00.326 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:00.323+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:00.336 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:00.336 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:00.336 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:00.336 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:00.336 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:00.351 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:00.427 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:00.423+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:00.513 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002840205, 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.514 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:00.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:00.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:00.518 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002781575, 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002758695000001, 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:00.522 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:00.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:00.526 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002740632, 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:00.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:00.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002703145, 2024-09-22T01:16:00.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:00.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:00.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002677522999999, 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:00.534 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002656675000001, 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.538 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:00.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:00.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002639436000001, 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:00.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:00.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:00.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002587342, 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:00.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002567469000001, 2024-09-22T01:16:00.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:00.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:00.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002521221, 2024-09-22T01:16:00.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:00.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:00.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002500533999999, 2024-09-22T01:16:00.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002451845, 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:00.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002382792000001, 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:00.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:00.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002362958000001, 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:00.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:00.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002314327000001, 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002293267000001, 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002242704, 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002221777000001, 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:00.589 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 14.002193889999999, 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:00.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:00.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:00.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:00.593 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:00.595 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:00.592+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:00.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:00.609 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:00.695 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:00.691+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:00.746 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:00.747 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:00.763 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:00.796 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:00.792+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:00.897 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:00.895+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:00.918 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:00.919 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:00.919 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:00.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:00.998 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:00.992+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:01.099 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:01.096+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.157578644000001, 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.001729343000001, 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.001708013999998, 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.001244940999999, 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.001218071, 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.001194562999999, 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.000997616999999, 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.000969551000001, 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999728112, 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999690346000001, 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999666392000002, 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999645306000001, 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:01.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999624841999999, 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:01.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999604820999998, 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:01.167 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:01.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999585623000002, 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:01.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:01.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999564510999999, 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:01.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:01.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999543919000001, 2024-09-22T01:16:01.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:01.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:01.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999522557999999, 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:01.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:01.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:01.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999501244000001, 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:01.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:01.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.999482226000001, 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:01.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.197 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:01.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:01.200 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:01.196+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:01.301 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:01.299+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:01.326 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:01.326 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:01.326 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:01.339 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:01.424 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:01.423+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:01.503 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:01.499+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:01.505 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:01.505 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:01.505 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:01.505 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:01.505 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:01.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:01.603 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:01.600+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:01.704 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:01.699+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:01.723 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:01.723 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 130.73706809699999, 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:01.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:01.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:01.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:01.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531320437999995, 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:01.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:01.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531287665999997, 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:01.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531266201999998, 2024-09-22T01:16:01.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:01.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531245259000002, 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.744 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:01.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:01.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531225351000003, 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:01.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:01.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531204970000005, 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:01.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:01.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.531183033000005, 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:01.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.530673452000002, 2024-09-22T01:16:01.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:01.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.799292405000003, 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:01.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.799258101000007, 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:01.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:01.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.799187457000002, 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:01.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:01.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.799035752999998, 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:01.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.798820215999996, 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.798606991, 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.798408867000006, 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.796 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.798205074999998, 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:01.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:01.800 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.797954250000004, 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:01.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:01.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 66.797735230000001, 2024-09-22T01:16:01.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:01.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:01.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.110697622, 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:01.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:01.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:01.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:01.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:01.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:01.816 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:01.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:01.818 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:01.819 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:01.812+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:01.918 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:01.915+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:01.984 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:01.985 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:02.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:02.006 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:02.007 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:02.023 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:02.024 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:02.024 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:02.019+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:02.040 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:02.040 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:02.056 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:02.057 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:02.077 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:02.077 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:02.113 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:02.113 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:02.120 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:02.116+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:02.144 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.144 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.147 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:02.147 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:02.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:02.174 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:02.174 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:02.221 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:02.216+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:02.304 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.304 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:02.322 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:02.319+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:02.423 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:02.419+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:02.443 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.443 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:02.544 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:02.543+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:02.625 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:02.620+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:02.632 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.632 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.644 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:16:02.648 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:02.724 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:02.720+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:02.818 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.818 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.824 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:02.823+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:02.834 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:02.926 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:02.920+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:02.962 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:02.963 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:02.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:03.027 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:03.023+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:03.127 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:03.124+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:03.147 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:03.147 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:03.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:03.228 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:03.227+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:03.326 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:03.326 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:03.329 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:03.327+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:03.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:03.430 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:03.427+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:03.458 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:03.458 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:03.473 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:03.530 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:03.528+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:03.598 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:03.599 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:03.612 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:03.627 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:16:03.630 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:03.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:03.630 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-22T01:16:03.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:16:03.635 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:03.631+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:03.732 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:03.728+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:03.787 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:03.788 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:03.802 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:03.847 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:03.844+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:03.933 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:03.931+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 123, 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:16:03.620108+0000", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:37.510453+0000", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5801984, 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "noscrub", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:16:03.956 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "luminous", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:16:03.957 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:03.958 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:03.959 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:03.960 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:03.961 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:03.962 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:16:03.963 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:03.964 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:16:03.965 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:03.966 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:03.967 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:03.968 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 5, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 5, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.75, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 3.75, 2024-09-22T01:16:03.969 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 3.75, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:03.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:03.971 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "123", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 54, 2024-09-22T01:16:03.972 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 123, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:03.973 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:03.974 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:03.975 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:03.976 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.978 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.979 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:03.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:03.981 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:03.982 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:03.985 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.988 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:03.990 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:16:03.995 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:03.996 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.997 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:03.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 0.4185333251953125, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 118, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:04.002 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:16:04.006 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:04.007 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:16:04.008 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:16:04.009 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:16:04.012 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:04.013 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 112, 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:04.018 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0.2866363525390625, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 95, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:04.019 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.020 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.022 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:16:04.023 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.024 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.026 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:16:04.027 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:04.028 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:04.029 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [ 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.1", 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: 4, 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: 7 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:16:04.030 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [ 2024-09-22T01:16:04.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 51, 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 4 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [ 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:04.032 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 51, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [], 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:16:04.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:04.034 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:04.035 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:04.035 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:04.036 INFO:tasks.thrashosds.thrasher:j is {'epoch': 123, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:16:03.620108+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:37.510453+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 123, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4185333251953125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 0.2866363525390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [4, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 4}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:16:04.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:16:04.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:04.037 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:04.032+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:04.135 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:04.131+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:04.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:04.204 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:04.204 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:04.218 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:04.508 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:04.509 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:04.509 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:04.509 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:04.510 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:04.510 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:16:04.512 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:16:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:04.513 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:04.514 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-22T01:16:03.629250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1473658269,"num_objects":2290,"num_object_clones":585,"num_object_copies":4580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":977,"num_read":254854,"num_read_kb":956817,"num_write":21526,"num_write_kb":379361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1107,"num_bytes_recovered":85611109,"num_keys_recovered":234,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18197,"ondisk_log_size":18197,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":849220,"kb_used_data":491404,"kb_used_omap":12,"kb_used_meta":357619,"kb_avail":754125500,"statfs":{"total":773094113280,"available":772224512000,"internally_reserved":0,"allocated":503197696,"data_stored":676178217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":366202416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-162191983,"num_objects":-328,"num_object_clones":-152,"num_object_copies":-656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-328,"num_whiteouts":-121,"num_read":27324,"num_read_kb":36977,"num_write":395,"num_write_kb":1747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029904"},"pg_stats":[{"pgid":"3.1f","version":"121'301","reported_seq":2876,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.632190+0000","last_change":"2024-09-22T01:16:02.632724+0000","last_active":"2024-09-22T01:16:02.632190+0000","last_peered":"2024-09-22T01:16:02.632190+0000","last_clean":"2024-09-22T01:16:02.632190+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:16:02.632190+0000","last_undegraded":"2024-09-22T01:16:02.632190+0000","last_fullsized":"2024-09-22T01:16:02.632190+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:39:52.743448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022142289999999999,"stat_sum":{"num_bytes":19676866,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2442,"num_read_kb":21975,"num_write":280,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.1e","version":"122'481","reported_seq":12371,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.781818+0000","last_change":"2024-09-22T01:16:02.633054+0000","last_active":"2024-09-22T01:16:02.781818+0000","last_peered":"2024-09-22T01:16:02.781818+0000","last_clean":"2024-09-22T01:16:02.781818+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:16:02.781818+0000","last_undegraded":"2024-09-22T01:16:02.781818+0000","last_fullsized":"2024-09-22T01:16:02.781818+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:35:46.606182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029510999999999999,"stat_sum":{"num_bytes":53408224,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":28,"num_read":12536,"num_read_kb":37108,"num_write":515,"num_write_kb":16351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.1d","version":"122'379","reported_seq":19868,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.653487+0000","last_change":"2024-09-22T01:16:02.653487+0000","last_active":"2024-09-22T01:16:02.653487+0000","last_peered":"2024-09-22T01:16:02.653487+0000","last_clean":"2024-09-22T01:16:02.653487+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:16:02.653487+0000","last_undegraded":"2024-09-22T01:16:02.653487+0000","last_fullsized":"2024-09-22T01:16:02.653487+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:56:56.128713+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0082637189999999992,"stat_sum":{"num_bytes":32435997,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":24,"num_read":20508,"num_read_kb":27943,"num_write":363,"num_write_kb":10276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.1c","version":"122'540","reported_seq":12117,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.794084+0000","last_change":"2024-09-22T01:16:02.626709+0000","last_active":"2024-09-22T01:16:02.794084+0000","last_peered":"2024-09-22T01:16:02.794084+0000","last_clean":"2024-09-22T01:16:02.794084+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:16:02.794084+0000","last_undegraded":"2024-09-22T01:16:02.794084+0000","last_fullsized":"2024-09-22T01:16:02.794084+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:44.870152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002455806,"stat_sum":{"num_bytes":48471435,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":29,"num_read":12722,"num_read_kb":38484,"num_write":634,"num_write_kb":12877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.1b","version":"121'349","reported_seq":5360,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-22T01:16:01.665406+0000","last_change":"2024-09-22T01:16:01.665969+0000","last_active":"2024-09-22T01:16:01.665406+0000","last_peered":"2024-09-22T01:16:01.665406+0000","last_clean":"2024-09-22T01:16:01.665406+0000","last_became_active":"2024-09-22T01:15:32.940539+0000","last_became_peered":"2024-09-22T01:15:32.940539+0000","last_unstale":"2024-09-22T01:16:01.665406+0000","last_undegraded":"2024-09-22T01:16:01.665406+0000","last_fullsized":"2024-09-22T01:16:01.665406+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:45:45.143096+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.048069015999999999,"stat_sum":{"num_bytes":50824394,"num_objects":66,"num_object_clones":16,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":5155,"num_read_kb":11294,"num_write":292,"num_write_kb":4889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"35","length":"2"}]},{"pgid":"3.1a","version":"121'1049","reported_seq":16352,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.802033+0000","last_change":"2024-09-22T01:16:02.626674+0000","last_active":"2024-09-22T01:16:02.802033+0000","last_peered":"2024-09-22T01:16:02.802033+0000","last_clean":"2024-09-22T01:16:02.802033+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:16:02.802033+0000","last_undegraded":"2024-09-22T01:16:02.802033+0000","last_fullsized":"2024-09-22T01:16:02.802033+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:31.328067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001830187,"stat_sum":{"num_bytes":68306763,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":23,"num_read":19431,"num_read_kb":77256,"num_write":1619,"num_write_kb":27218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.19","version":"121'402","reported_seq":5893,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.624203+0000","last_change":"2024-09-22T01:16:02.624671+0000","last_active":"2024-09-22T01:16:02.624203+0000","last_peered":"2024-09-22T01:16:02.624203+0000","last_clean":"2024-09-22T01:16:02.624203+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:16:02.624203+0000","last_undegraded":"2024-09-22T01:16:02.624203+0000","last_fullsized":"2024-09-22T01:16:02.624203+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:31.536998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001002515,"stat_sum":{"num_bytes":63750893,"num_objects":76,"num_object_clones":20,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":34,"num_read":6788,"num_read_kb":27882,"num_write":329,"num_write_kb":6644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.18","version":"121'328","reported_seq":6105,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.632873+0000","last_change":"2024-09-22T01:16:02.633389+0000","last_active":"2024-09-22T01:16:02.632873+0000","last_peered":"2024-09-22T01:16:02.632873+0000","last_clean":"2024-09-22T01:16:02.632873+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:16:02.632873+0000","last_undegraded":"2024-09-22T01:16:02.632873+0000","last_fullsized":"2024-09-22T01:16:02.632873+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:19:04.413308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028420239999999999,"stat_sum":{"num_bytes":52139626,"num_objects":70,"num_object_clones":15,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":35,"num_read":5865,"num_read_kb":39218,"num_write":312,"num_write_kb":21840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.17","version":"122'462","reported_seq":5138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.830862+0000","last_change":"2024-09-22T01:16:02.631228+0000","last_active":"2024-09-22T01:16:02.830862+0000","last_peered":"2024-09-22T01:16:02.830862+0000","last_clean":"2024-09-22T01:16:02.830862+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:16:02.830862+0000","last_undegraded":"2024-09-22T01:16:02.830862+0000","last_fullsized":"2024-09-22T01:16:02.830862+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:36.169789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010798889999999999,"stat_sum":{"num_bytes":67247509,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":27,"num_read":6090,"num_read_kb":34107,"num_write":466,"num_write_kb":11874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.16","version":"121'454","reported_seq":12698,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.625086+0000","last_change":"2024-09-22T01:16:02.625697+0000","last_active":"2024-09-22T01:16:02.625086+0000","last_peered":"2024-09-22T01:16:02.625086+0000","last_clean":"2024-09-22T01:16:02.625086+0000","last_became_active":"2024-09-22T01:14:41.576276+0000","last_became_peered":"2024-09-22T01:14:41.576276+0000","last_unstale":"2024-09-22T01:16:02.625086+0000","last_undegraded":"2024-09-22T01:16:02.625086+0000","last_fullsized":"2024-09-22T01:16:02.625086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:28:29.597349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018155809999999999,"stat_sum":{"num_bytes":47003754,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":42,"num_read":12251,"num_read_kb":27167,"num_write":472,"num_write_kb":13004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4954128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.15","version":"121'747","reported_seq":3531,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-22T01:16:01.639930+0000","last_change":"2024-09-22T01:16:01.641571+0000","last_active":"2024-09-22T01:16:01.639930+0000","last_peered":"2024-09-22T01:16:01.639930+0000","last_clean":"2024-09-22T01:16:01.639930+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:16:01.639930+0000","last_undegraded":"2024-09-22T01:16:01.639930+0000","last_fullsized":"2024-09-22T01:16:01.639930+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:48:35.703089+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.023476174999999998,"stat_sum":{"num_bytes":22018777,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":5450,"num_read_kb":18607,"num_write":1123,"num_write_kb":4370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"35","length":"2"}]},{"pgid":"3.14","version":"121'650","reported_seq":9984,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.646610+0000","last_change":"2024-09-22T01:16:02.647117+0000","last_active":"2024-09-22T01:16:02.646610+0000","last_peered":"2024-09-22T01:16:02.646610+0000","last_clean":"2024-09-22T01:16:02.646610+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:16:02.646610+0000","last_undegraded":"2024-09-22T01:16:02.646610+0000","last_fullsized":"2024-09-22T01:16:02.646610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:54:15.093032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023726620000000002,"stat_sum":{"num_bytes":65747428,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":31,"num_read":11784,"num_read_kb":35239,"num_write":851,"num_write_kb":12765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.7","version":"0'0","reported_seq":195,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.645806+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:16:02.645806+0000","last_peered":"2024-09-22T01:16:02.645806+0000","last_clean":"2024-09-22T01:16:02.645806+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:16:02.645806+0000","last_undegraded":"2024-09-22T01:16:02.645806+0000","last_fullsized":"2024-09-22T01:16:02.645806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"122'497","reported_seq":3867,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.625369+0000","last_change":"2024-09-22T01:16:02.625369+0000","last_active":"2024-09-22T01:16:02.625369+0000","last_peered":"2024-09-22T01:16:02.625369+0000","last_clean":"2024-09-22T01:16:02.625369+0000","last_became_active":"2024-09-22T01:15:59.611357+0000","last_became_peered":"2024-09-22T01:15:59.611357+0000","last_unstale":"2024-09-22T01:16:02.625369+0000","last_undegraded":"2024-09-22T01:16:02.625369+0000","last_fullsized":"2024-09-22T01:16:02.625369+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:18:28.663019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001447247,"stat_sum":{"num_bytes":35342781,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":25,"num_read":5941,"num_read_kb":40716,"num_write":491,"num_write_kb":23279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.623439+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:16:02.623439+0000","last_peered":"2024-09-22T01:16:02.623439+0000","last_clean":"2024-09-22T01:16:02.623439+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:16:02.623439+0000","last_undegraded":"2024-09-22T01:16:02.623439+0000","last_fullsized":"2024-09-22T01:16:02.623439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"121'298","reported_seq":3774,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.722576+0000","last_change":"2024-09-22T01:16:02.723611+0000","last_active":"2024-09-22T01:16:02.722576+0000","last_peered":"2024-09-22T01:16:02.722576+0000","last_clean":"2024-09-22T01:16:02.722576+0000","last_became_active":"2024-09-22T01:15:32.926099+0000","last_became_peered":"2024-09-22T01:15:32.926099+0000","last_unstale":"2024-09-22T01:16:02.722576+0000","last_undegraded":"2024-09-22T01:16:02.722576+0000","last_fullsized":"2024-09-22T01:16:02.722576+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:08:18.648209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003692318,"stat_sum":{"num_bytes":33998487,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":32,"num_read":3880,"num_read_kb":26227,"num_write":265,"num_write_kb":9851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.645598+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:16:02.645598+0000","last_peered":"2024-09-22T01:16:02.645598+0000","last_clean":"2024-09-22T01:16:02.645598+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:16:02.645598+0000","last_undegraded":"2024-09-22T01:16:02.645598+0000","last_fullsized":"2024-09-22T01:16:02.645598+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"121'550","reported_seq":2995,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.721278+0000","last_change":"2024-09-22T01:16:02.722395+0000","last_active":"2024-09-22T01:16:02.721278+0000","last_peered":"2024-09-22T01:16:02.721278+0000","last_clean":"2024-09-22T01:16:02.721278+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:16:02.721278+0000","last_undegraded":"2024-09-22T01:16:02.721278+0000","last_fullsized":"2024-09-22T01:16:02.721278+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:21:39.762311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026321510000000001,"stat_sum":{"num_bytes":29917963,"num_objects":86,"num_object_clones":21,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":37,"num_read":4980,"num_read_kb":19058,"num_write":709,"num_write_kb":6444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.4","version":"0'0","reported_seq":216,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.629917+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:16:02.629917+0000","last_peered":"2024-09-22T01:16:02.629917+0000","last_clean":"2024-09-22T01:16:02.629917+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:16:02.629917+0000","last_undegraded":"2024-09-22T01:16:02.629917+0000","last_fullsized":"2024-09-22T01:16:02.629917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"122'1078","reported_seq":8194,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.624850+0000","last_change":"2024-09-22T01:16:02.624850+0000","last_active":"2024-09-22T01:16:02.624850+0000","last_peered":"2024-09-22T01:16:02.624850+0000","last_clean":"2024-09-22T01:16:02.624850+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:16:02.624850+0000","last_undegraded":"2024-09-22T01:16:02.624850+0000","last_fullsized":"2024-09-22T01:16:02.624850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1078,"log_dups_size":0,"ondisk_log_size":1078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:20:34.130219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001647176,"stat_sum":{"num_bytes":43098504,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":11325,"num_read_kb":39535,"num_write":1671,"num_write_kb":16616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.3","version":"121'634","reported_seq":5977,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-22T01:16:01.811129+0000","last_change":"2024-09-22T01:16:01.697469+0000","last_active":"2024-09-22T01:16:01.811129+0000","last_peered":"2024-09-22T01:16:01.811129+0000","last_clean":"2024-09-22T01:16:01.811129+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:16:01.811129+0000","last_undegraded":"2024-09-22T01:16:01.811129+0000","last_fullsized":"2024-09-22T01:16:01.811129+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:10:57.647576+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.078642235000000005,"stat_sum":{"num_bytes":35706355,"num_objects":89,"num_object_clones":25,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":34,"num_read":6928,"num_read_kb":29339,"num_write":723,"num_write_kb":11271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"35","length":"2"}]},{"pgid":"2.2","version":"23'2","reported_seq":199,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.644124+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:16:02.644124+0000","last_peered":"2024-09-22T01:16:02.644124+0000","last_clean":"2024-09-22T01:16:02.644124+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:16:02.644124+0000","last_undegraded":"2024-09-22T01:16:02.644124+0000","last_fullsized":"2024-09-22T01:16:02.644124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"121'424","reported_seq":3734,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.631416+0000","last_change":"2024-09-22T01:16:02.631975+0000","last_active":"2024-09-22T01:16:02.631416+0000","last_peered":"2024-09-22T01:16:02.631416+0000","last_clean":"2024-09-22T01:16:02.631416+0000","last_became_active":"2024-09-22T01:15:51.529181+0000","last_became_peered":"2024-09-22T01:15:51.529181+0000","last_unstale":"2024-09-22T01:16:02.631416+0000","last_undegraded":"2024-09-22T01:16:02.631416+0000","last_fullsized":"2024-09-22T01:16:02.631416+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:10:21.988921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016790640000000001,"stat_sum":{"num_bytes":38875661,"num_objects":68,"num_object_clones":13,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":4043,"num_read_kb":28009,"num_write":446,"num_write_kb":8216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.629717+0000","last_change":"2024-09-22T01:15:48.063999+0000","last_active":"2024-09-22T01:16:02.629717+0000","last_peered":"2024-09-22T01:16:02.629717+0000","last_clean":"2024-09-22T01:16:02.629717+0000","last_became_active":"2024-09-22T01:15:48.063387+0000","last_became_peered":"2024-09-22T01:15:48.063387+0000","last_unstale":"2024-09-22T01:16:02.629717+0000","last_undegraded":"2024-09-22T01:16:02.629717+0000","last_fullsized":"2024-09-22T01:16:02.629717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:03:32.378683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"122'430","reported_seq":6174,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.767130+0000","last_change":"2024-09-22T01:16:02.767605+0000","last_active":"2024-09-22T01:16:02.767130+0000","last_peered":"2024-09-22T01:16:02.767130+0000","last_clean":"2024-09-22T01:16:02.767130+0000","last_became_active":"2024-09-22T01:15:32.937217+0000","last_became_peered":"2024-09-22T01:15:32.937217+0000","last_unstale":"2024-09-22T01:16:02.767130+0000","last_undegraded":"2024-09-22T01:16:02.767130+0000","last_fullsized":"2024-09-22T01:16:02.767130+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:31:48.121179+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0099972359999999996,"stat_sum":{"num_bytes":28504892,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":29,"num_read":7218,"num_read_kb":16454,"num_write":424,"num_write_kb":11598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.0","version":"0'0","reported_seq":195,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.643728+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:16:02.643728+0000","last_peered":"2024-09-22T01:16:02.643728+0000","last_clean":"2024-09-22T01:16:02.643728+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:16:02.643728+0000","last_undegraded":"2024-09-22T01:16:02.643728+0000","last_fullsized":"2024-09-22T01:16:02.643728+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"121'548","reported_seq":5788,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.624046+0000","last_change":"2024-09-22T01:16:02.624546+0000","last_active":"2024-09-22T01:16:02.624046+0000","last_peered":"2024-09-22T01:16:02.624046+0000","last_clean":"2024-09-22T01:16:02.624046+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:16:02.624046+0000","last_undegraded":"2024-09-22T01:16:02.624046+0000","last_fullsized":"2024-09-22T01:16:02.624046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:53.547815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092053100000000004,"stat_sum":{"num_bytes":85750894,"num_objects":77,"num_object_clones":17,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":34,"num_read":7503,"num_read_kb":25304,"num_write":627,"num_write_kb":9978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"2.3","version":"21'1","reported_seq":217,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.644085+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:16:02.644085+0000","last_peered":"2024-09-22T01:16:02.644085+0000","last_clean":"2024-09-22T01:16:02.644085+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:16:02.644085+0000","last_undegraded":"2024-09-22T01:16:02.644085+0000","last_fullsized":"2024-09-22T01:16:02.644085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":368,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.623875+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:16:02.623875+0000","last_peered":"2024-09-22T01:16:02.623875+0000","last_clean":"2024-09-22T01:16:02.623875+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:16:02.623875+0000","last_undegraded":"2024-09-22T01:16:02.623875+0000","last_fullsized":"2024-09-22T01:16:02.623875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"121'404","reported_seq":3376,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.624312+0000","last_change":"2024-09-22T01:16:02.624639+0000","last_active":"2024-09-22T01:16:02.624312+0000","last_peered":"2024-09-22T01:16:02.624312+0000","last_clean":"2024-09-22T01:16:02.624312+0000","last_became_active":"2024-09-22T01:15:51.530462+0000","last_became_peered":"2024-09-22T01:15:51.530462+0000","last_unstale":"2024-09-22T01:16:02.624312+0000","last_undegraded":"2024-09-22T01:16:02.624312+0000","last_fullsized":"2024-09-22T01:16:02.624312+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:45:32.833181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013540799999999999,"stat_sum":{"num_bytes":62995073,"num_objects":73,"num_object_clones":24,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":30,"num_read":3546,"num_read_kb":17164,"num_write":316,"num_write_kb":4195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.9","version":"121'808","reported_seq":18314,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.645186+0000","last_change":"2024-09-22T01:16:02.645640+0000","last_active":"2024-09-22T01:16:02.645186+0000","last_peered":"2024-09-22T01:16:02.645186+0000","last_clean":"2024-09-22T01:16:02.645186+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:16:02.645186+0000","last_undegraded":"2024-09-22T01:16:02.645186+0000","last_fullsized":"2024-09-22T01:16:02.645186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:42.446361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016378650000000001,"stat_sum":{"num_bytes":93056327,"num_objects":89,"num_object_clones":21,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":32,"num_read":22878,"num_read_kb":79997,"num_write":839,"num_write_kb":23075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.a","version":"122'548","reported_seq":3520,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.647751+0000","last_change":"2024-09-22T01:16:02.647751+0000","last_active":"2024-09-22T01:16:02.647751+0000","last_peered":"2024-09-22T01:16:02.647751+0000","last_clean":"2024-09-22T01:16:02.647751+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:16:02.647751+0000","last_undegraded":"2024-09-22T01:16:02.647751+0000","last_fullsized":"2024-09-22T01:16:02.647751+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:27.502216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003114627,"stat_sum":{"num_bytes":27862245,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":6472,"num_read_kb":11160,"num_write":536,"num_write_kb":8451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.b","version":"122'482","reported_seq":4369,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.831771+0000","last_change":"2024-09-22T01:16:02.726884+0000","last_active":"2024-09-22T01:16:02.831771+0000","last_peered":"2024-09-22T01:16:02.831771+0000","last_clean":"2024-09-22T01:16:02.831771+0000","last_became_active":"2024-09-22T01:15:50.529119+0000","last_became_peered":"2024-09-22T01:15:50.529119+0000","last_unstale":"2024-09-22T01:16:02.831771+0000","last_undegraded":"2024-09-22T01:16:02.831771+0000","last_fullsized":"2024-09-22T01:16:02.831771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:46.175333+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0071567109999999996,"stat_sum":{"num_bytes":42767142,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":27,"num_read":5181,"num_read_kb":17722,"num_write":614,"num_write_kb":9231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.c","version":"121'590","reported_seq":5445,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-22T01:16:01.670021+0000","last_change":"2024-09-22T01:16:01.670021+0000","last_active":"2024-09-22T01:16:01.670021+0000","last_peered":"2024-09-22T01:16:01.670021+0000","last_clean":"2024-09-22T01:16:01.670021+0000","last_became_active":"2024-09-22T01:15:51.537372+0000","last_became_peered":"2024-09-22T01:15:51.537372+0000","last_unstale":"2024-09-22T01:16:01.670021+0000","last_undegraded":"2024-09-22T01:16:01.670021+0000","last_fullsized":"2024-09-22T01:16:01.670021+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:51:49.456093+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.051903075999999999,"stat_sum":{"num_bytes":36122539,"num_objects":99,"num_object_clones":32,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":47,"num_read":5535,"num_read_kb":28779,"num_write":691,"num_write_kb":10030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"35","length":"2"}]},{"pgid":"3.d","version":"122'606","reported_seq":2495,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.649263+0000","last_change":"2024-09-22T01:16:02.649263+0000","last_active":"2024-09-22T01:16:02.649263+0000","last_peered":"2024-09-22T01:16:02.649263+0000","last_clean":"2024-09-22T01:16:02.649263+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:16:02.649263+0000","last_undegraded":"2024-09-22T01:16:02.649263+0000","last_fullsized":"2024-09-22T01:16:02.649263+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:37:24.167825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038786390000000001,"stat_sum":{"num_bytes":22115661,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":31,"num_read":3803,"num_read_kb":12092,"num_write":804,"num_write_kb":6520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.e","version":"121'744","reported_seq":3492,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.625145+0000","last_change":"2024-09-22T01:16:02.625389+0000","last_active":"2024-09-22T01:16:02.625145+0000","last_peered":"2024-09-22T01:16:02.625145+0000","last_clean":"2024-09-22T01:16:02.625145+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:16:02.625145+0000","last_undegraded":"2024-09-22T01:16:02.625145+0000","last_fullsized":"2024-09-22T01:16:02.625145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:34:19.313752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013083909999999999,"stat_sum":{"num_bytes":34534375,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":27,"num_read":5060,"num_read_kb":21170,"num_write":997,"num_write_kb":11670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.f","version":"121'619","reported_seq":3594,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-22T01:16:01.765888+0000","last_change":"2024-09-22T01:16:01.682558+0000","last_active":"2024-09-22T01:16:01.765888+0000","last_peered":"2024-09-22T01:16:01.765888+0000","last_clean":"2024-09-22T01:16:01.765888+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:16:01.765888+0000","last_undegraded":"2024-09-22T01:16:01.765888+0000","last_fullsized":"2024-09-22T01:16:01.765888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:23.248103+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.063860421000000001,"stat_sum":{"num_bytes":32141127,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":4857,"num_read_kb":23108,"num_write":760,"num_write_kb":11284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"35","length":"2"}]},{"pgid":"3.10","version":"121'549","reported_seq":3144,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.624758+0000","last_change":"2024-09-22T01:16:02.625420+0000","last_active":"2024-09-22T01:16:02.624758+0000","last_peered":"2024-09-22T01:16:02.624758+0000","last_clean":"2024-09-22T01:16:02.624758+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:16:02.624758+0000","last_undegraded":"2024-09-22T01:16:02.624758+0000","last_fullsized":"2024-09-22T01:16:02.624758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:41.877247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019295950000000001,"stat_sum":{"num_bytes":34975941,"num_objects":72,"num_object_clones":16,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":29,"num_read":4304,"num_read_kb":15438,"num_write":623,"num_write_kb":5252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.11","version":"121'801","reported_seq":4485,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.645947+0000","last_change":"2024-09-22T01:16:02.646343+0000","last_active":"2024-09-22T01:16:02.645947+0000","last_peered":"2024-09-22T01:16:02.645947+0000","last_clean":"2024-09-22T01:16:02.645947+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:16:02.645947+0000","last_undegraded":"2024-09-22T01:16:02.645947+0000","last_fullsized":"2024-09-22T01:16:02.645947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:50:38.259673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020222460000000001,"stat_sum":{"num_bytes":47660024,"num_objects":72,"num_object_clones":20,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":28,"num_read":7096,"num_read_kb":50369,"num_write":1025,"num_write_kb":22529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.12","version":"121'706","reported_seq":4685,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.720092+0000","last_change":"2024-09-22T01:16:02.721111+0000","last_active":"2024-09-22T01:16:02.720092+0000","last_peered":"2024-09-22T01:16:02.720092+0000","last_clean":"2024-09-22T01:16:02.720092+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:16:02.720092+0000","last_undegraded":"2024-09-22T01:16:02.720092+0000","last_fullsized":"2024-09-22T01:16:02.720092+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:28:28.325865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001437937,"stat_sum":{"num_bytes":57588570,"num_objects":77,"num_object_clones":19,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":36,"num_read":6317,"num_read_kb":26427,"num_write":870,"num_write_kb":11318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"34","length":"3"}]},{"pgid":"3.13","version":"121'619","reported_seq":5179,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-22T01:16:02.645021+0000","last_change":"2024-09-22T01:16:02.646271+0000","last_active":"2024-09-22T01:16:02.645021+0000","last_peered":"2024-09-22T01:16:02.645021+0000","last_clean":"2024-09-22T01:16:02.645021+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:16:02.645021+0000","last_undegraded":"2024-09-22T01:16:02.645021+0000","last_fullsized":"2024-09-22T01:16:02.645021+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:27:32.453218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022667939999999999,"stat_sum":{"num_bytes":59021655,"num_objects":84,"num_object_clones":21,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":38,"num_read":6849,"num_read_kb":32369,"num_write":678,"num_write_kb":7616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"34","length":"3"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1473067882,"num_objects":2286,"num_object_clones":585,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2286,"num_whiteouts":977,"num_read":254738,"num_read_kb":956717,"num_write":21365,"num_write_kb":376943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1102,"num_bytes_recovered":85611090,"num_keys_recovered":234,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":490278912,"data_stored":669317966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18077,"ondisk_log_size":18077,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41412,"kb_used_data":6064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330428,"statfs":{"total":96636764160,"available":96594358272,"internally_reserved":0,"allocated":6209536,"data_stored":6841390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89732,"kb_used_data":44656,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94282108,"statfs":{"total":96636764160,"available":96544878592,"internally_reserved":0,"allocated":45727744,"data_stored":56204400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114936,"kb_used_data":69852,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94256904,"statfs":{"total":96636764160,"available":96519069696,"internally_reserved":0,"allocated":71528448,"data_stored":93458688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95016,"kb_used_data":67984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94276824,"statfs":{"total":96636764160,"available":96539467776,"internally_reserved":0,"allocated":69615616,"data_stored":97885940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133460,"kb_used_data":80056,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94238380,"statfs":{"total":96636764160,"available":96500101120,"internally_reserved":0,"allocated":81977344,"data_stored":113902834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131708,"kb_used_data":78312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94240132,"statfs":{"total":96636764160,"available":96501895168,"internally_reserved":0,"allocated":80191488,"data_stored":107094004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131236,"kb_used_data":82000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94240604,"statfs":{"total":96636764160,"available":96502378496,"internally_reserved":0,"allocated":83968000,"data_stored":114526187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542175,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":111720,"kb_used_data":62480,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94260120,"statfs":{"total":96636764160,"available":96522362880,"internally_reserved":0,"allocated":63979520,"data_stored":86264774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":62541824,"data_stored":85592142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81924096,"data_stored":113256931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":78741504,"data_stored":106415116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":80424960,"data_stored":113033891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":68165632,"data_stored":97207052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":69484544,"data_stored":92189432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44236800,"data_stored":55460900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4759552,"data_stored":6162502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:16:04.517 INFO:tasks.thrashosds.thrasher:Setting 3.16 to [2, 0] 2024-09-22T01:16:04.517 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.16', '2', '0'] 2024-09-22T01:16:04.517 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.16 2 0 2024-09-22T01:16:04.518 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:04.518 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:04.519 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:04.519 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:04.519 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:04.520 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:04.520 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:04.521 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:04.521 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:04.507+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:04.611 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:04.608+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:04.644 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422973000000001, 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.645 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:04.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.649 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422942920000001, 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.650 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:04.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422921689000001, 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422900425, 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422879118000001, 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422857477999999, 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422812348000001, 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422781323000001, 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422757284999999, 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422736021, 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422683988999999, 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.703 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:04.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422625018, 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.706 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.711 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.422521250999999, 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:04.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:04.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:04.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.716 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421941469, 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.717 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.718 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:04.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.721 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421821026, 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:04.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421741025999999, 2024-09-22T01:16:04.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421276838000001, 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:04.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.735 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:04.736 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421207262999999, 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.740 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421140513999999, 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.741 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:04.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 12.421068896, 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:04.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:04.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:04.750 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:04.751 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:04.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:04.752 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:04.747+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:04.853 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:04.848+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:04.925 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:04.925 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:04.925 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:04.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:04.953 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:04.951+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:05.054 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:05.048+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:05.101 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:05.102 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:05.102 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:05.102 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:05.102 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:05.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:05.155 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:05.151+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:05.256 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:05.251+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771356242, 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:05.281 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:05.282 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:05.283 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:05.284 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771297612000001, 2024-09-22T01:16:05.285 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.286 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:05.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:05.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771274731999998, 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.289 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:05.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771256669, 2024-09-22T01:16:05.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.295 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771219181999999, 2024-09-22T01:16:05.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.77119356, 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.301 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:05.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:05.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:05.304 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771172711999998, 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:05.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:05.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771155473, 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.309 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:05.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:05.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:05.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771103378999999, 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.313 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:05.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.315 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771083506, 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.317 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:05.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:05.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:05.321 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771037258, 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:05.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.323 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:05.325 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.771016571000001, 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:05.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:05.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:05.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:05.329 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770967882000001, 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:05.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:05.333 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770898829, 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.337 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770878995, 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770830363999998, 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770809304, 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:05.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770758741000002, 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:05.353 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770737814, 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:05.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:05.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:05.357 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 18.770709926999999, 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:05.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:05.361 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:05.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:05.363 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:05.359+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:05.464 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:05.460+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:05.489 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:05.490 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:05.490 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:05.490 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:05.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:05.565 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:05.563+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:05.619 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:05.619 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:05.632 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:05.646 INFO:teuthology.orchestra.run.smithi096.stderr:set 3.16 pg_upmap mapping to [2,0] 2024-09-22T01:16:05.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:16:05.678 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:05.675+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:05.767 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:05.764+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:05.786 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:05.786 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:05.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:05.786 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:05.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.841046601999999, 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:05.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:05.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:05.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.685197300999999, 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:05.794 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.798 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.685175972, 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:05.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.684712899000001, 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:05.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:05.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:05.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.684686029000002, 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:05.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.684662521, 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:05.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.684465575000001, 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:05.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:05.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:05.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:05.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.684437508999999, 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:05.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:05.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:05.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683196070000001, 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:05.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:05.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683158303999999, 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:05.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:05.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:05.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.68313435, 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:05.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.841 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683113263999999, 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:05.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:05.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:05.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683092800000001, 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:05.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:05.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:05.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.852 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683072779, 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.853 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:05.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:05.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683053580999999, 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:05.857 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:05.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683032469, 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:05.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:05.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.866 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.683011876999998, 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:05.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:05.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.871 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.682990516, 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.872 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.682969201999999, 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:05.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.682950183999999, 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:05.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:05.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:05.886 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:05.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:05.888 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:05.883+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:05.988 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:05.987+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:06.048 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:06.048 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:06.048 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:06.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:06.104 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:06.103+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:06.191 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:06.188+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:06.221 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:06.221 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:06.221 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:06.221 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:06.221 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:06.236 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:06.291 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:06.287+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:06.359 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 135.37104266599999, 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:06.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165295006999997, 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:06.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:06.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:06.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:06.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165262235, 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:06.372 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:06.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165240771000001, 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:06.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:06.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:06.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165219828000005, 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165199920000006, 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:06.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:06.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165179538999993, 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.165157601999994, 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:06.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:06.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:06.397 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.164648021000005, 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:06.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:06.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:06.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.433266974000006, 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:06.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:06.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.433232669999995, 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.433162026000005, 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:06.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:06.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.433010322000001, 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:06.414 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.432794784999999, 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.432581560000003, 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.432383435999995, 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:06.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.432179644000001, 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.431928819000007, 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:06.439 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 71.431709799000004, 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.744672190999999, 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:06.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:06.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:06.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:06.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:06.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:06.451 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:06.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:06.453 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:06.451+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:06.522 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:06.592 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:06.591+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:06.621 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:06.621 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:06.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:16:06.654 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:06.651+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:06.696 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:06.696 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:06.715 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:06.715 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:06.733 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:06.733 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:06.752 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:06.753 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:06.755 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:06.752+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:06.788 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:06.788 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:06.791 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:06.791 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:06.806 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:06.812 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:06.813 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:06.835 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:06.835 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:06.855 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:06.851+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:06.856 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:06.856 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:06.918 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:06.919 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:06.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:06.972 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:06.971+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:07.060 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:07.055+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:07.086 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.086 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:07.188 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:07.187+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:07.257 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.257 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.262 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:07.256+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:07.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:07.362 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:07.356+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:07.433 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.433 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:07.463 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:07.459+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:07.564 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:07.560+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:07.579 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.579 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:07.665 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:07.663+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:07.766 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:07.760+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:07.773 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.773 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.789 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:07.875 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:07.872+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:07.956 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:07.957 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:07.967 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:07.963+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:07.971 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:08.068 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:08.063+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:08.137 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.137 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:08.169 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:08.167+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:08.270 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:08.264+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:08.272 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.272 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.286 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:08.370 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:08.367+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:08.451 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.452 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.467 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:08.511 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:08.508+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:08.572 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:08.568+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:08.588 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.588 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.602 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:08.672 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:08.671+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:08.681 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:16:08.773 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.773 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.773 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:08.771+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:08.788 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:08.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:08.873 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:08.871+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:08.907 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:08.907 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:08.922 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:08.974 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:08.972+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:09.013 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:09.013 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:09.029 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:09.030 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:09.042 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:09.042 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:09.046 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:09.046 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:09.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:09.065 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:09.065 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:09.075 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:09.071+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:09.083 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:09.083 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:09.103 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:09.104 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:09.123 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:09.123 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:09.144 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:09.145 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:09.204 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:09.203+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:09.224 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:09.224 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:09.224 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:09.224 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:09.224 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:09.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:09.277 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:09.275+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:09.365 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854519083, 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.366 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:09.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854460453000002, 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.374 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854437572999998, 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.378 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.85441951, 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854382022999999, 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:09.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.386 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854356401, 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:09.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:09.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:09.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854335552999999, 2024-09-22T01:16:09.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:09.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854318314, 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.394 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.395 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.85426622, 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.398 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:09.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:09.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854246347, 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.402 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:09.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.405 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854200099, 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:09.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:09.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854179412000001, 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:09.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854130723000001, 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.414 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:09.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:09.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.85406167, 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:09.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.421 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.854041836, 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.853993204999998, 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:09.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.853972145, 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:09.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:09.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.853921582000002, 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.853900655, 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.437 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 22.853872767999999, 2024-09-22T01:16:09.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:09.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:09.445 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:09.446 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:09.440+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:09.446 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:09.516 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:16:09.547 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:09.544+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:09.593 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:09.594 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:09.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:09.648 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:09.644+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:09.729 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:09.729 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:09.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:09.792 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:09.791+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:09.850 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:09.844+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:09.865 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.918905670000001, 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:09.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:09.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:09.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.763056368999997, 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:09.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:09.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.763035039999998, 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:09.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.762571967, 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.762545097, 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:09.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:09.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.762521589000002, 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.762324642999999, 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:09.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:09.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:09.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.762296577000001, 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:09.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:09.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.761055138000003, 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:09.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:09.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.761017371999998, 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760993417999998, 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:09.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760972332000001, 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.920 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:09.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760951867999999, 2024-09-22T01:16:09.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760931847000002, 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760912648999998, 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:09.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760891536999999, 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:09.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760870945000001, 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:09.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:09.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760849583999999, 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:09.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760828269999998, 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:09.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.760809252000001, 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:09.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:09.960 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:09.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:09.962 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:09.960+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:10.062 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:10.059+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:10.127 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:10.127 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:10.127 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:10.127 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:10.141 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:10.163 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:10.160+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:10.264 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:10.260+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:10.308 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:10.308 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:10.308 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:10.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:10.365 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:10.360+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:10.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 139.46072640099999, 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:10.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:10.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:10.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:10.455 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254978742000006, 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:10.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254945969999994, 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:10.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254924505999995, 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254903562999999, 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:10.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:10.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254883655, 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:10.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254863274000002, 2024-09-22T01:16:10.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:10.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:10.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:10.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254841337000002, 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:10.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:10.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.254331755999999, 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:10.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522950709, 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:10.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:10.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:10.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522916405000004, 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:10.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522845760999999, 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522694056999995, 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:10.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:10.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522478519999993, 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522265294999997, 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.522067171000003, 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:10.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.521863378999996, 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:10.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:10.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.521612554000001, 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 75.521393533999998, 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:10.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:10.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:10.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.834355926000001, 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:10.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:10.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:10.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:10.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:10.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:10.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:10.539 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:10.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:10.542 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:10.536+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:10.641 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:10.636+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:10.674 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:10.674 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:10.674 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:10.674 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:10.678 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:10.678 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:10.678 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-22T01:16:10.679 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-22T01:16:10.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:10.742 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:10.736+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:10.842 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:10.840+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:10.859 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:10.859 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:10.859 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:10.859 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:10.859 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:10.878 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:10.943 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:10.940+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:11.005 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.106678969000001, 2024-09-22T01:16:11.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.010 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.106514613000002, 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:11.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101822837, 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.015 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:11.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:11.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101785282999998, 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:11.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:11.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101753121000002, 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:11.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:11.027 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101730783999997, 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:11.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101709424000006, 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.101688233999994, 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:11.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097782741000003, 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:11.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:11.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097760559999998, 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.045 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:11.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:11.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097739282000006, 2024-09-22T01:16:11.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:11.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:11.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097716934999994, 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097695259000005, 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.058 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.097673837000002, 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.091035091999998, 2024-09-22T01:16:11.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:11.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:11.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.090818345000002, 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.071 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:11.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:11.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:11.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.090435717999995, 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:11.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:11.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.090030178000006, 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.083 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.086813716999998, 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:11.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:11.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.087 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.050028322000003, 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:11.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:11.092 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:11.092 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:11.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:11.094 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:11.088+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:11.162 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:11.193 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:11.192+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:11.219 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:11.219 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:11.219 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:11.219 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:11.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:11.294 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:11.288+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:11.306 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:11.323 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:11.323 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:11.339 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:11.352 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:11.353 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:11.353 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:11.353 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:11.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:11.358 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:11.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:11.377 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:11.377 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:11.395 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:11.392+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:11.398 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:11.398 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:11.420 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:11.420 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:11.440 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:11.440 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:11.482 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:11.483 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:11.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:11.497 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:11.491+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:11.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:16:11.596 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:11.592+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:11.630 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:11.630 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:11.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:11.697 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:11.695+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:11.702 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-22T01:16:11.699+0000 7f33f8d74640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-22T01:16:11.708 INFO:teuthology.orchestra.run.smithi096.stderr:osd set-full-ratio 0.001 2024-09-22T01:16:11.817 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:11.817 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:11.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:11.902 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:11.895+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:11.902 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:11.795+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:11.954 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:11.954 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:11.969 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:11.999 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:11.995+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:12.091 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.092 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.100 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:12.096+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:12.106 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:12.201 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:12.199+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:12.270 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.270 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.284 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:12.301 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:12.299+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:12.402 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:12.399+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:12.407 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.407 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:12.503 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:12.499+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:12.542 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.542 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.557 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:12.604 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:12.600+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:12.674 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.674 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.689 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:12.722 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:12.735 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:12.732+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:12.738 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-22T01:16:12.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-22T01:16:12.806 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:12.800+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:12.861 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:12.862 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:12.877 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:12.906 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:12.903+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:13.007 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:13.004+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:13.043 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:13.043 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:13.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:13.147 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:13.144+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:13.209 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:13.207+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:13.224 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:13.224 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:13.238 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:13.327 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:13.324+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:13.402 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:13.402 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:13.410 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:13.408+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:13.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:13.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:13.512 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:13.507+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:13.551 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:13.551 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:13.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:13.611 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:13.611 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:13.612 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:13.608+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:13.630 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:13.630 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:13.647 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:13.664 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:13.664 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:13.684 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:13.685 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:13.705 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:13.706 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:13.713 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:13.711+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:osd set-full-ratio 0.95 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:13.733 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:13.734 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:13.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:13.757 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:13.757 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:13.820 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:13.819+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.941018462000002, 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.785169160999999, 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.785147832, 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.784684759000001, 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.784657889000002, 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.784634381000004, 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.784437435000001, 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.784409369000002, 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783167929999998, 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783130163999999, 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.78310621, 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783085124000003, 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783064660000001, 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783044639000003, 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783025440999999, 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.783004329000001, 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.782983737000002, 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.782962376, 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:13.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.782941061999999, 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:13.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:13.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:13.971 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.782922044000003, 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:13.972 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:13.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:13.976 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:13.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:13.978 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:13.976+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:14.078 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:14.072+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:14.152 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:14.152 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:14.152 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:14.166 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:14.212 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:14.211+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:14.280 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:14.276+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:14.288 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:14.288 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:14.288 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:14.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:14.388 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:14.387+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:14.471 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 143.48315996400001, 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:14.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:14.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:14.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:14.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:14.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277412304999999, 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:14.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277379533000001, 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:14.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:14.554 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277358069000002, 2024-09-22T01:16:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:14.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:14.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277337126000006, 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:14.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277317217999993, 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:14.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:14.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:14.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:14.572 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277296836999994, 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:14.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:14.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.277274899999995, 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:14.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:14.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 81.276765319000006, 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:14.581 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:14.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:14.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.545384272000007, 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.586 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:14.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.545349967999996, 2024-09-22T01:16:14.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:14.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:14.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.545279324000006, 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:14.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.599 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.545127620000002, 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.544912083, 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.544698858000004, 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.544500733999996, 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:14.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.544296942000003, 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:14.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:14.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:14.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:14.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.544046116999994, 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:14.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 79.543827097000005, 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:14.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.856789489000001, 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:14.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:14.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:14.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:14.641 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:14.642 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:14.643 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:14.639+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:14.743 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:14.739+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:14.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:16:14.821 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:14.821 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:14.821 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:14.821 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:14.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:14.884 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:14.883+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:14.945 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:14.943+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:14.957 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:14.957 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:14.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:15.046 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:15.040+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.235984767000005, 2024-09-22T01:16:15.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:15.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:15.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.235820411000006, 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:15.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:15.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.231128635000005, 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:15.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.231091081000002, 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.152 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.231058919000006, 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:15.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.231036582000002, 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:15.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.231015221999996, 2024-09-22T01:16:15.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:15.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.230994031999998, 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.166 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:15.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:15.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.227088538999993, 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:15.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:15.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:15.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.227066358000002, 2024-09-22T01:16:15.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:15.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:15.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.227045079999996, 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:15.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.227022732999998, 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:15.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:15.187 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.227001056999995, 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:15.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:15.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.226979635000006, 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.220340890000003, 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:15.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.200 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.220124143000007, 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:15.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:15.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.219741515999999, 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.219335975999996, 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.216119515000003, 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:15.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:15.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.179334119999993, 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:15.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:15.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.223 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:15.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:15.225 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:15.223+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:15.325 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:15.323+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:15.399 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:15.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:15.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:15.426 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:15.420+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:15.527 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:15.523+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:15.585 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:15.585 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:15.585 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:15.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:15.640 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:15.639+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:15.715 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:16:15.729 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:15.727+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:15.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 140.75474047099999, 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:15.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:15.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:15.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.207176675, 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:15.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.511541438999998, 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:15.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.511525862999999, 2024-09-22T01:16:15.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:15.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.511510035000001, 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:15.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:15.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.511495042, 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:15.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.511479852000001, 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.509588262000001, 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.509568411, 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.509550968999999, 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.509535514, 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.50951448, 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.505852532999999, 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.505835928, 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.505820567000001, 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.505805403, 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.501443470000002, 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.501427975999999, 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.501412434999999, 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 20.497323866999999, 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:15.869 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:15.869 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:15.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:15.871 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:15.867+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:15.972 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:15.967+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:16.006 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:16.006 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:16.006 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:16.006 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:16.020 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:16.067 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:16.067 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:16.074 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:16.072+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:16.084 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:16.084 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:16.100 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:16.117 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:16.117 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:16.135 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:16.136 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:16.143 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.143 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.154 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:16.155 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:16.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:16.175 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:16.175 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:16.195 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:16.196 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:16.199 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:16.196+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:16.277 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:16.275+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:16.316 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.316 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.330 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:16.378 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:16.375+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:16.447 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.447 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.462 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:16.479 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:16.475+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:16.580 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:16.576+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:16.583 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.583 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.597 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:16.680 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:16.675+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:16.774 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.774 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.780 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:16.779+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:16.790 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:16.881 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:16.876+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:16.921 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:16.921 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:16.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:17.023 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:17.020+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:17.084 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:17.079+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:17.102 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.102 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.116 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:17.185 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:17.183+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:17.282 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.283 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.285 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:17.283+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:17.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:17.387 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:17.383+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:17.417 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.418 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:17.487 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:17.484+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:17.554 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.554 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.568 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:17.588 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:17.583+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:17.689 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:17.684+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:17.736 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.736 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.746 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:16:17.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:17.792 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:17.791+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:17.890 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:17.887+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:17.931 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:17.931 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:17.949 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:17.992 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:17.991+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:18.092 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:18.088+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:18.115 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:18.115 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:18.129 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:18.193 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:18.192+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:18.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:18.245 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:18.245 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:18.261 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:18.308 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:18.307+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:18.351 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:18.351 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:18.368 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:18.368 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:18.385 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:18.385 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:18.395 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:18.392+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:18.402 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:18.402 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:18.422 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:18.423 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:18.426 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:18.427 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:18.427 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:18.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:18.443 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:18.443 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:18.464 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:18.464 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:18.484 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:18.485 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:18.495 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:18.491+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:18.559 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:18.560 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:18.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:18.596 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:18.592+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 147.710671203, 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:18.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:18.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504923543999993, 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:18.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:18.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:18.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504890771999996, 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:18.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504869307999996, 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504848365000001, 2024-09-22T01:16:18.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504828457000002, 2024-09-22T01:16:18.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:18.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504808076000003, 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:18.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504786139000004, 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.729 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:18.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 85.504276558000001, 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772895511000002, 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:18.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772861207000005, 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:18.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772790563000001, 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772638858999997, 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772423321999995, 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772210096999999, 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.772011973000005, 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.771808180999997, 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.771557356000002, 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.771338335999999, 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.084300727999999, 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:18.786 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:18.786 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:18.695+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:18.787 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:18.788 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:16:18.788 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:18.788 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:18.789 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-22T01:16:18.789 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:16:18.789 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:16:18.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-22T01:16:18.829 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:18.827+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:18.897 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:18.895+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:18.958 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:18.958 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:16:18.959 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:18.965 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:18.965 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:18.965 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:18.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:19.024 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:19.023+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:19.099 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:19.096+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:19.142 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:19.143 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:19.143 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:19.156 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:19.200 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:19.199+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.278 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.37904743, 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.378883074000001, 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374191298, 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:19.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:19.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374153743999997, 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:19.291 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.295 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374121582000001, 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374099244999996, 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:19.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:19.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374077885000005, 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.374056694999993, 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370151202000002, 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370129020999997, 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370107743000005, 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370085395999993, 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370063720000005, 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.370042298000001, 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.363403552999998, 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.363186806000002, 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.362804178999994, 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.362398639000006, 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.359182177999998, 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 84.322396783000002, 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.362 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:19.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:19.364 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:19.359+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:19.464 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:19.463+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:203501 3.9 3:97c45993:::rbd_header.1124999addd0:head [call rbd.snapshot_remove in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139)", 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:16:19.526144+0000", 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.012292042, 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012366551, 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 203501 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526144+0000", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526144+0000", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526147+0000", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5530000000000001e-06 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526165+0000", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7024000000000002e-05 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526204+0000", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199000000000002e-05 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526229+0000", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5449e-05 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526326+0000", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6698000000000002e-05 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-09-22T01:16:19.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.526334+0000", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2660000000000001e-06 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.530191+0000", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038567369999999998 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.530436+0000", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024466800000000002 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.531767+0000", 2024-09-22T01:16:19.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013317030000000001 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:19.535357+0000", 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035900429999999998 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-22T01:16:19.544 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:19.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:19.565 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:19.560+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:19.666 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:19.660+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:19.727 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:19.727 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:19.728 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:19.728 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:19.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:19.793 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:19.791+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 144.84280134900001, 2024-09-22T01:16:19.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:19.872 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.29523755300001, 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:19.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.599602316999999, 2024-09-22T01:16:19.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:19.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:19.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.599586741, 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:19.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:19.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.599570913000001, 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.59955592, 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.599540730000001, 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:19.898 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.597649140000001, 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.899 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:19.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.597629289, 2024-09-22T01:16:19.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:19.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.597611847, 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:19.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:19.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.597596392, 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:19.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:19.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:19.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.597575358, 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.593913410999999, 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:19.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.593896806, 2024-09-22T01:16:19.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:19.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.593881445000001, 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:19.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:19.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.593866281, 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:19.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:19.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.589504347999998, 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:19.937 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:19.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:19.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:19.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.589488853999999, 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:19.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.589473312999999, 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.949 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.585384744999999, 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:19.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:19.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:19.954 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:19.955 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:19.956 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:19.952+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:20.056 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:20.052+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:20.088 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:20.089 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:20.089 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:20.089 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:20.102 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:20.158 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:20.155+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:20.258 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:20.252+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:20.267 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:20.267 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:20.267 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:20.267 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:20.267 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:20.281 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:20.359 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:20.355+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:20.440 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.632653340000005, 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.441 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:20.445 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 67.425264001000002, 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.446 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.450 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752655183000002, 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:20.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752635052999999, 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.455 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752614222999998, 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:20.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752594198000001, 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752573517999998, 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:20.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752551371999999, 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:20.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.752014760000002, 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.751990110000001, 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.481 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:20.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:20.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:20.485 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.751953745000002, 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:20.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:20.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.749004376999999, 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747396412000001, 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:20.494 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:20.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:20.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747375399999999, 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:20.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:20.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747346378, 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.503 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:20.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747314029999998, 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:20.507 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:20.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:20.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747247909999999, 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.512 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:20.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.747217502000002, 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:20.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.745720878, 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:20.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 20.745690517, 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:20.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:20.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:20.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:20.528 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:20.529 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:20.530 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:20.531 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:20.524+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:20.631 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:20.628+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:20.693 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:20.693 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:20.709 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:20.709 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:20.709 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:20.709 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:20.710 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:20.710 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:20.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:20.728 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:20.728 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:20.731 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:20.727+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:20.744 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:20.744 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:20.764 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:20.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:20.784 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:20.784 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:20.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:16:20.801 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:20.802 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:20.820 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:20.820 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:20.832 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:20.828+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:20.890 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:20.890 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:20.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:20.932 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:20.931+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:21.034 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:21.028+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:21.074 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.074 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:21.135 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:21.131+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:21.235 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:21.231+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:21.248 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.248 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:21.336 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:21.332+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:21.384 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.384 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.398 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:21.439 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:21.436+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:21.538 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:16:21.539 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:21.532+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:21.563 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.563 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:21.639 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:21.635+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:21.740 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:21.736+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:21.756 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.756 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.771 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:21.842 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:21.839+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:21.938 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:21.938 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:21.942 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:21.935+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:21.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:22.043 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:22.039+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:22.121 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.121 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:22.183 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:22.179+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:22.244 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:22.239+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:22.266 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.267 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:22.369 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:22.367+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:22.442 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.442 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.446 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:22.443+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:22.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:22.547 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:22.543+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:22.632 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.632 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.648 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:22.649 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:22.643+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:22.749 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:22.747+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:22.781 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.781 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:22.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:22.850 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:22.843+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:22.951 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:22.947+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:22.957 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:22.957 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:22.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:23.029 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:23.030 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:23.047 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:23.047 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:23.051 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:23.047+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:23.064 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:23.064 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:23.080 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:23.080 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:23.101 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:23.101 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:23.102 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:23.102 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 152.11655362100001, 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:23.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:23.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:23.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:23.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:23.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:23.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910805961999998, 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:23.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.91077319, 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910751726000001, 2024-09-22T01:16:23.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910730783000005, 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:23.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:23.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.125 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910710875000007, 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:23.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:23.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:23.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910690493999994, 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:23.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910668556999994, 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:23.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:23.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:23.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 89.910158976000005, 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178777929000006, 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:23.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:23.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178743624999996, 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:23.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:23.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178672981000005, 2024-09-22T01:16:23.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:23.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178521277000002, 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:23.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178305739999999, 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.178092515000003, 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:23.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:23.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.177894390999995, 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:23.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:23.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.171 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.177690599000002, 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:23.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.177439774000007, 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.177220754000004, 2024-09-22T01:16:23.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:23.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.490183146, 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:23.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:23.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.190 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:23.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:23.192 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:23.192 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:23.192 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:23.193 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:23.193 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:23.194 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:23.194 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:23.187+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:23.292 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:23.287+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:23.360 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:23.361 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:23.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:23.393 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:23.387+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:23.493 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:23.491+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:23.539 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:23.540 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:23.562 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:23.567 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:16:23.594 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:23.587+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:23.695 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:23.691+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.840611252000002, 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.742 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.840446896000003, 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:23.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:23.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:23.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835755120000002, 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835717566, 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:23.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:23.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835685404000003, 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:23.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835663066999999, 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:23.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835641706999994, 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:23.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:23.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:23.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.835620516999995, 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:23.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:23.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831715024000005, 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:23.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:23.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:23.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831692842999999, 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:23.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831671564999994, 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831649217999995, 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831627542000007, 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:23.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:23.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.831606120000004, 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.795 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:23.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.824967375, 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:23.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.824750628000004, 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.804 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:23.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.824368000999996, 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.823962460999994, 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:23.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:23.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.820746, 2024-09-22T01:16:23.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:23.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:23.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 88.783960605000004, 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.822 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:23.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:23.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:23.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:23.826 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:23.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:23.828 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:23.823+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:23.929 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:23.923+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:23.976 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:23.976 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:23.977 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:16:23.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:16:23.993 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:23.994 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:24.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:24.052 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:24.051+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:24.131 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:24.127+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:24.171 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:24.171 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:24.171 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:24.171 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:24.171 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:24.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:24.232 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:24.227+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:16:24.278 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 143, 2024-09-22T01:16:24.278 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:16:23.538262+0000", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:37.510453+0000", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5804032, 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "nodeep-scrub", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.279 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "luminous", 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:16:24.280 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:24.281 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:24.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:24.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:24.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:16:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:24.286 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:24.287 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:24.288 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:24.289 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:24.290 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 5, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 5, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.75, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 3.75, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 3.75, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:24.292 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:24.293 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:24.294 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "143", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 54, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 143, 2024-09-22T01:16:24.295 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:24.296 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:24.297 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:24.298 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.299 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.300 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:24.305 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.308 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.310 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 128, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.316 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.319 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:16:24.320 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 0.4185333251953125, 2024-09-22T01:16:24.321 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 118, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.323 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.327 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:16:24.331 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.332 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 112, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.333 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.334 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:16:24.335 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:16:24.336 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:24.337 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0.2866363525390625, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 95, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.339 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:16:24.340 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:24.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.344 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.346 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.347 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [ 2024-09-22T01:16:24.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.1", 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: 4, 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: 7 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.16", 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: 2, 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: 0 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.349 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:16:24.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [ 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 32, 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 12 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [ 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.351 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 32, 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [], 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:16:24.352 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:16:24.353 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:16:24.353 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:16:24.353 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:16:24.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:24.355 INFO:tasks.thrashosds.thrasher:j is {'epoch': 143, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:16:23.538262+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:37.510453+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4185333251953125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 0.2866363525390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [4, 7]}, {'pgid': '3.16', 'osds': [2, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 32, 'length': 12}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 32, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:16:24.355 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-22T01:16:24.355 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:24.347+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 149.336393453, 2024-09-22T01:16:24.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:24.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:24.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 109.78882965699999, 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:24.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:24.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.093194421, 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:24.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:24.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:24.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.093178845000001, 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:24.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:24.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:24.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.093163016999998, 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:24.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:24.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.093148024000001, 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:24.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.093132833999999, 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:24.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:24.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:24.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.091241243999999, 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.091221393000001, 2024-09-22T01:16:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.091203951000001, 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.091188496000001, 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.091167462000001, 2024-09-22T01:16:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.087505515, 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.087488910000001, 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.087473548999998, 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.087458385000001, 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.083096451999999, 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.083080958, 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.083065417, 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.078976849, 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:24.450 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:24.451 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:24.493 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:24.491+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:24.521 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:24.556 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:24.551+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:24.591 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:24.591 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:24.591 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:24.591 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:24.605 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:24.691 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:24.687+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:24.758 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:24.755+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:24.765 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:24.765 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:24.765 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:24.765 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:24.765 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:24.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:24.865 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:24.863+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:24.951 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:24.952 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.142419937, 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:24.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 71.935030597999997, 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.957 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:24.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:24.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.26242178, 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:24.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:24.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:24.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.262401650000001, 2024-09-22T01:16:24.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:24.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.262380820000001, 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.262360794999999, 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:24.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:24.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.262340115000001, 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:24.979 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:24.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:24.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.262317969000001, 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:24.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:24.987 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.261781357, 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.988 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.989 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:24.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.261756707, 2024-09-22T01:16:24.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:24.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:24.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.261720342, 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:24.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:24.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:24.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:25.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.258770974000001, 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.001 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.257163008999999, 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.005 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:25.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:25.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.257141997000002, 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.257112974999998, 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.257080627000001, 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:25.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:25.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.257014507000001, 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:25.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:25.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:25.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.256984099, 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:25.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.255487474999999, 2024-09-22T01:16:25.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:25.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:25.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 25.255457113999999, 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:25.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:25.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:25.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:25.041 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:25.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:25.041 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:25.007+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:25.109 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:25.103+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:25.169 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:25.170 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:25.170 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:25.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:25.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:25.240 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:25.239+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:25.311 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:25.307+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:25.348 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:25.349 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:25.352 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:25.352 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:25.352 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:25.352 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:25.353 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:25.365 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:25.365 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:25.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:25.381 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:25.382 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:25.398 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:25.398 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:25.412 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:25.407+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:25.418 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:25.438 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:25.438 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:25.456 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:25.456 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:25.474 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:25.475 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:25.537 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:25.535+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:25.539 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:25.539 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:25.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:25.612 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:25.607+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:25.685 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:25.685 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:25.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:25.743 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:25.739+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:25.815 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:25.811+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:25.864 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:25.864 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:25.879 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:25.927 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:25.923+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:26.001 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.001 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:26.017 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:26.015+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:26.118 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:26.111+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.191 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:26.235 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:26.231+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:26.321 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:26.315+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:26.357 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.357 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.371 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:26.420 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:26.419+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:26.488 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.489 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:26.521 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:26.515+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:26.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:16:26.622 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:26.615+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:26.646 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.646 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.663 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:26.723 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:26.719+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:26.802 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.802 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.817 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:26.864 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:26.863+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:26.925 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:26.923+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:26.980 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:26.980 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:26.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:27.025 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:27.019+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:27.126 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:27.123+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:27.157 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:27.157 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:27.171 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:27.256 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:27.255+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:27.328 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:27.323+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:27.338 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:27.338 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:27.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:27.429 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:27.423+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:27.492 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:27.522 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:27.522 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:27.530 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:27.527+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:27.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:27.630 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:27.627+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:27.633 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:27.633 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:27.650 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:27.650 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:27.666 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:27.666 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:27.683 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:27.683 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:27.703 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:27.703 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:27.703 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:27.704 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:27.704 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:27.721 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:27.731 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:27.731 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:27.732 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:27.727+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:27.753 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:27.754 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:27.776 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:27.776 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:27.777 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:16:27.832 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:27.827+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:27.858 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:27.858 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:27.858 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:27.871 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:27.933 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:27.927+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.095043176999994, 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:27.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:27.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.094878820999995, 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:27.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:27.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090187044999993, 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.002 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:28.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:28.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:28.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090149491000005, 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.006 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:28.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090117328999995, 2024-09-22T01:16:28.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:28.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:28.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090094992000004, 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:28.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:28.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090073631999999, 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.019 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.090052442000001, 2024-09-22T01:16:28.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:28.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.027 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086146948999996, 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086124768000005, 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.032 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:28.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:28.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086103489999999, 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:28.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:28.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086081143000001, 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:28.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086059466999998, 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.086038044999995, 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:28.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.079399300000006, 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:28.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:28.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.079182552999995, 2024-09-22T01:16:28.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:28.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:28.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.078799926000002, 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:28.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:28.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.078394385999999, 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.075177925000006, 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 93.038392529999996, 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.080 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:28.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:28.082 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:28.079+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:28.182 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:28.179+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:28.199 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:28.200 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:28.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:28.283 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:28.279+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:28.321 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:28.321 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:28.321 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:28.333 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:28.384 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:28.379+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:28.455 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 153.43235338100001, 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:28.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:28.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:28.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:28.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.462 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 113.88478958499999, 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:28.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:28.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:28.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.189154348999999, 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:28.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:28.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.189138773000003, 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:28.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.189122945000001, 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.189107952000001, 2024-09-22T01:16:28.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:28.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.189092762000001, 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:28.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.187201172000002, 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:28.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:28.494 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.187181320999997, 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:28.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:28.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:28.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.187163879000003, 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:28.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:28.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.187148424, 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:28.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.187127390000001, 2024-09-22T01:16:28.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:28.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.183465443000003, 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:28.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.183448837999997, 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.183433477000001, 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:28.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:28.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.183418312999997, 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:28.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.179056379999999, 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:28.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.179040886000003, 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:28.537 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.179025344999999, 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:28.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.174936776999999, 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:28.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:28.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:28.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:28.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:28.546 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:28.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:28.548 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:28.543+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:28.649 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:28.647+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:28.680 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:28.680 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:28.680 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:28.694 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:28.750 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:28.747+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:28.825 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:28.825 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:28.825 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:28.825 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:28.825 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:28.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:28.887 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:28.883+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:28.951 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:28.947+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.156670551000005, 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:28.966 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 75.949281212000002, 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:28.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:28.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276672393999998, 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:28.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:28.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:28.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:28.979 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276652263999999, 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.980 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.981 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.982 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.983 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.984 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276631433999999, 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.985 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.986 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.987 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.988 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276611409000001, 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.989 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.990 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.991 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.992 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276590728999999, 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:28.993 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.994 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:28.995 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:28.996 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:28.997 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276568583, 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:28.998 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:28.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:29.000 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.001 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276031970999998, 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:29.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:29.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.276007321000002, 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.275970955999998, 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.273021587999999, 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:29.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271413623000001, 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271392611, 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:29.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:29.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:29.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271363589, 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271331240999999, 2024-09-22T01:16:29.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:29.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271265120999999, 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:29.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.271234712999998, 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:29.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:29.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:29.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.269738089000001, 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:29.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 29.269707728, 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:29.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:29.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:29.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.053 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.054 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:29.054 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:29.051+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:29.154 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:29.151+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:29.225 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.225 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:29.225 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:29.225 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.239 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:29.255 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:29.251+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:29.356 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:29.356 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:29.356 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-22T01:16:29.356 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:16:29.356 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:16:29.356 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-22T01:16:29.357 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:29.351+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:29.406 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.406 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:29.407 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:29.407 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:29.407 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:29.457 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:29.455+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:29.487 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.487 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-22T01:16:29.488 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.541 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.541 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:29.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:29.541 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:29.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393970594999999, 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.542 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:29.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:29.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:29.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393870655000001, 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.546 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:29.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393801764000003, 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.550 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393725700999994, 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.554 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:29.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393662913, 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:29.558 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393031979, 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.562 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.393002839999994, 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.566 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:29.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:29.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:29.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392973514999994, 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.570 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:29.573 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392942879000003, 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.574 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:29.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.576 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.577 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392907210999994, 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:29.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:29.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392878228000001, 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:29.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392844015999998, 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392775196000002, 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.594 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392674682000006, 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:29.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392606528000002, 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:29.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:29.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392522798000002, 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.603 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:29.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392388920000002, 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:29.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392262071000005, 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392225116000006, 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.392173474000003, 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:29.619 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:29.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:29.623 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:29.625 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:29.623+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:29.725 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:29.719+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:29.798 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.799 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:29.799 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:29.799 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:29.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:16:29.813 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:29.844 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:29.843+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:29.926 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:29.919+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:29.958 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:29.959 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:29.973 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:29.973 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:29.973 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:29.973 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:29.974 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:29.976 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:29.976 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:29.987 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:29.992 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:29.993 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:30.009 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:30.009 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:30.028 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:30.029 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:30.035 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:30.031+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:30.048 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:30.049 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:30.068 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:30.069 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:30.086 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:30.089 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:30.128 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:30.123+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:30.167 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.167 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.183 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:30.229 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:30.227+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:30.309 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.309 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.323 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:30.330 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:30.327+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:30.431 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:30.427+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:30.439 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.439 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.454 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:30.532 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:30.527+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:30.575 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.575 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:30.632 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:30.627+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:30.722 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.722 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.733 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:30.731+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:30.736 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:30.796 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:16:30.834 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:30.827+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:30.882 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:30.882 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:30.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:30.944 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:30.943+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:31.021 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.021 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:31.036 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:31.031+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:31.137 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:31.135+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:31.218 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.218 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.232 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:31.238 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:31.231+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:31.339 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:31.335+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:31.401 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.401 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.415 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:31.439 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:31.435+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:31.538 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.538 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.539 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:31.535+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:31.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:31.642 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:31.639+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:31.720 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.721 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.735 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:31.742 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:31.735+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:31.843 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:31.839+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:31.912 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:31.912 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:31.927 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:31.973 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:31.971+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:32.044 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:32.039+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:32.054 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:32.054 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:32.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:32.105 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:32.145 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:32.139+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:32.241 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:32.241 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:32.246 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:32.239+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:32.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:32.258 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:32.259 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:32.276 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:32.276 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:32.293 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:32.293 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:32.310 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:32.310 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:32.331 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:32.331 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:32.347 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:32.343+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:32.352 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:32.352 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:32.371 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:32.371 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:32.371 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:32.371 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:32.371 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:32.373 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:32.373 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:32.386 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:32.396 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:32.396 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:32.449 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:32.443+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:32.521 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 157.49814573, 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:32.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:32.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.950581934, 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:32.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:32.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.254946697999998, 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:32.533 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:32.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:32.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.254931122000002, 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.254915294, 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.254900300999999, 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:32.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.254885111, 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:32.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.252993521, 2024-09-22T01:16:32.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:32.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.252973670000003, 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:32.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:32.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.252956228000002, 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:32.565 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.252940772999999, 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:32.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.252919738999999, 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:32.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:32.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.249257792000002, 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:32.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:32.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.249241187000003, 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:32.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:32.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.249225826, 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:32.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:32.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.249210662000003, 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:32.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:32.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.244848728999997, 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:32.596 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:32.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:32.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.244833235000002, 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:32.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.244817693999998, 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:32.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:32.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.240729125999998, 2024-09-22T01:16:32.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:32.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:32.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:32.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:32.614 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:32.615 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:32.615 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:32.611+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:32.716 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:32.711+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:32.749 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:32.749 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:32.750 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:32.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:32.838 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:16:32.851 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:32.847+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:32.917 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:32.915+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:32.942 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:32.942 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:32.942 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:32.942 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:32.942 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:32.958 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:33.043 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:33.039+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:33.119 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:33.115+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:33.130 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:33.130 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.320802807000007, 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.131 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:33.132 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.133 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.134 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 80.113413468000005, 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.135 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.136 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.137 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:33.138 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.139 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440804649999997, 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.140 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:33.141 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:33.142 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.143 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440784520000001, 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.144 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:33.145 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.146 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.147 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440763689999997, 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.148 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:33.149 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.150 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.151 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440743664999999, 2024-09-22T01:16:33.152 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.153 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:33.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:33.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440722985000001, 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:33.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:33.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.160 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440700839000002, 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.161 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:33.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:33.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440164226999997, 2024-09-22T01:16:33.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:33.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:33.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:33.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.169 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440139576999997, 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:33.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:33.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:33.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.440103211999997, 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:33.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:33.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:33.178 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.437153844000001, 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:33.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.180 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435545879000003, 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.183 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:33.186 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435524866999998, 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.187 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:33.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.191 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435495844999998, 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:33.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.195 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435463497000001, 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:33.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435397377000001, 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.200 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:33.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:33.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:33.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.435366969, 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.204 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:33.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:33.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.433870345000003, 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.208 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:33.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:33.212 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 33.433839984000002, 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:33.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.214 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.216 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:33.217 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:33.220 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:33.215+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:33.321 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:33.315+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:33.385 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:33.385 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:33.385 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:33.386 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:33.399 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:33.443 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:33.439+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:33.523 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:33.519+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:33.569 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:33.569 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:33.569 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:33.583 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:33.627 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:33.623+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:33.726 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:33.719+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:33.753 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:33.753 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:33.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:33.753 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.606279126000004, 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.757 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.606179186000006, 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.758 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:33.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:33.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.761 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.606110294999993, 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:33.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:33.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.606034231999999, 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605971444000005, 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605340510000005, 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:33.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.776 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605311370999999, 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:33.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.780 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605282045999999, 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.782 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605251409999994, 2024-09-22T01:16:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605215741999999, 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.788 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.791 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605186759000006, 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.794 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605152547000003, 2024-09-22T01:16:33.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:33.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.605083726999993, 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.799 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:33.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604983212999997, 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:33.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604915059000007, 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:33.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.810 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604831329000007, 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:33.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:33.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:33.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604697451000007, 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:33.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.818 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604570601999995, 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:33.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:33.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604533646999997, 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:33.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.604482004999994, 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:33.826 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:33.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:33.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:33.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:33.830 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:33.831 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:33.832 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:33.827+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:33.844 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:16:33.931 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:33.927+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:34.009 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:34.010 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:34.010 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:34.010 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:34.024 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:34.032 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:34.027+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:34.133 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:34.127+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:34.194 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:34.195 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:34.210 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:34.234 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:34.231+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:34.334 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:34.331+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:34.379 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:34.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159352331999997, 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:34.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:34.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:34.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:34.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.385 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159322252000003, 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:34.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:34.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159301020999997, 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.390 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:34.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.394 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159279757, 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:34.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:34.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159258450000003, 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.399 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:34.403 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159236810000003, 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:34.407 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159191679999999, 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:34.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159160655000001, 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:34.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:34.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159136617000001, 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.416 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159115352999997, 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.420 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:34.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159063320999998, 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:34.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.159004349999996, 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:34.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:34.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.158900582999998, 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.158320801000002, 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:34.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.158200358000002, 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:34.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:34.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:34.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:34.446 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.158120357999998, 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:34.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:34.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:34.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.450 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.157656170000003, 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:34.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:34.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.157586594999998, 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:34.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:34.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.157519846, 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:34.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.157448228, 2024-09-22T01:16:34.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:34.468 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:34.469 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:34.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:34.471 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:34.463+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:34.503 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:34.503 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:34.503 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-22T01:16:34.503 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-22T01:16:34.503 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-22T01:16:34.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-22T01:16:34.570 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:34.563+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:34.634 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:34.634 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:34.652 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:34.652 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "" 2024-09-22T01:16:34.652 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:34.653 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:34.653 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:34.653 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:34.653 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:34.654 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:34.670 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:34.671 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:34.671 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:34.667+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:34.672 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:34.690 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:34.690 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:34.708 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:34.708 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:34.726 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:34.726 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:34.743 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:34.764 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:34.765 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:34.772 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:34.767+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:34.835 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:34.836 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:34.850 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:34.872 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:34.871+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:34.972 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:34.973 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:34.973 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:34.971+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:34.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:35.074 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:35.067+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:35.116 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:35.117 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:35.133 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:35.176 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:35.175+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:35.257 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:35.445 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:35.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:35.447 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:35.443+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:35.547 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:35.543+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:35.632 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:35.632 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:35.646 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:35.692 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:35.691+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:35.749 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:35.747+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:35.823 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:35.823 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:35.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:35.849 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:35.843+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:35.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:16:35.950 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:35.943+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:36.005 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.005 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.019 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:36.051 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.047+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.133 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.133 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.148 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:36.152 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.147+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.253 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:36.251+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:36.264 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.264 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:36.354 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.347+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.401 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.401 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.415 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:36.455 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:36.451+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:36.535 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:36.539 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.539 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.556 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.551+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:36.657 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:36.651+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:36.693 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.693 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:36.758 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:36.755+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:36.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:36.833 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.833 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.848 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:36.860 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.855+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.924 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:36.924 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:36.941 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:36.941 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:36.957 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:36.958 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:36.960 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:36.955+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:36.964 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:36.964 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:36.974 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:36.974 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:36.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:36.995 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:36.995 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:37.015 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:37.015 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:37.033 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:37.053 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:37.061 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:37.055+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:37.117 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:37.117 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:37.117 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:37.117 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:37.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:37.161 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:37.159+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:37.262 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:37.259+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:37.297 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:37.297 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:37.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:37.363 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:37.359+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:37.432 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 92.622835014000003, 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 84.415445675000001, 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:37.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:37.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:37.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742836857, 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742816726999997, 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742795897000001, 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742775872000003, 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.455 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:37.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742755191999997, 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:37.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:37.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:37.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742733045999998, 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:37.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742196434, 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:37.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:37.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:37.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:37.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742171784, 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:37.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:37.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:37.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:37.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.742135419, 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.476 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.739186050999997, 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.480 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:37.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:37.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737578085999999, 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:37.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:37.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737557074000001, 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:37.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737528052000002, 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.493 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.494 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:37.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737495703999997, 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737429583999997, 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:37.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.737399175999997, 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.506 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:37.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:37.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.735902551999999, 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.510 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.512 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:37.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 37.735872190999999, 2024-09-22T01:16:37.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:37.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:37.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:37.518 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:37.520 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:37.520 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:37.515+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:37.621 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:37.619+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:37.695 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:37.695 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:37.695 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:37.695 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:37.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:37.722 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:37.719+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:37.823 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:37.819+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:37.829 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:37.829 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:37.829 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:37.830 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:37.830 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:37.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:37.924 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:37.919+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:38.008 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.008 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.860080498000002, 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.009 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859980558000004, 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.013 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:38.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:38.016 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859911667000006, 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:38.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:38.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859835603999997, 2024-09-22T01:16:38.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:38.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:38.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859772816000003, 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.024 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:38.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:38.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:38.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859141882000003, 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.028 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859112742999997, 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:38.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859083417999997, 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.035 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:38.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859052782000006, 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.039 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.042 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.859017113999997, 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.043 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:38.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858988131000004, 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.046 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:38.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:38.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858953919000001, 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:38.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:38.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.053 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858885099000005, 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:38.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:38.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858784584999995, 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:38.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:38.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:38.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858716431000005, 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:38.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.064 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858632701000005, 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:38.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:38.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:38.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858498823000005, 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.068 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858371973999994, 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858335018999995, 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:38.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:38.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:38.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:38.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.858283377000006, 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:38.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.082 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.084 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:38.084 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:38.079+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:38.186 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:38.179+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:38.253 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.253 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:38.253 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:38.253 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.267 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:38.286 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:38.283+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:38.386 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:38.383+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:38.439 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.439 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:38.439 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:38.439 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:38.439 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.452 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:38.499 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:38.495+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:38.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:16:38.576 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.576 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:38.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:38.576 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356707800999999, 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:38.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356677720999997, 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.581 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:38.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:38.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356656489999999, 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:38.585 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.589 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356635226000002, 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:38.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:38.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:38.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356613918999997, 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.594 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:38.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356592278999997, 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.598 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:38.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:38.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:38.602 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356547149000001, 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.603 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:38.604 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:38.605 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.606 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356516124000002, 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.607 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.608 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.609 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:38.610 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356492086000003, 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.611 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:38.612 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:38.613 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.614 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356470821999999, 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.615 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:38.616 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.617 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.618 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.619 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356418789999999, 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.620 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:38.621 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:38.622 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.623 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356359818999998, 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.624 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.625 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.626 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:38.627 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.356256051999999, 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.628 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.629 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.630 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.355676270000004, 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:38.631 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.632 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.633 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:38.634 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.355555827000003, 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:38.635 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.636 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:38.637 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:38.638 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.355475826999999, 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.639 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.640 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.641 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.642 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.355011638999997, 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.643 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.644 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.645 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.646 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.354942063999999, 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.647 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.648 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:38.649 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:38.650 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.354875315000001, 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.651 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.652 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:38.653 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:38.654 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 46.354803697000001, 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:38.655 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:38.656 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:38.657 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:38.658 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:38.659 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:38.661 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:38.655+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:38.760 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:38.759+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.837 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:38.837 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:38.837 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.851 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:38.862 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:38.859+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:38.963 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:38.959+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:38.968 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:38.968 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:38.968 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:38.968 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:38.969 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:38.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:39.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:39.075 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:39.071+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:39.154 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:39.154 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:39.154 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:39.154 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:39.154 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.643047946999999, 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.155 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.156 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.157 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:39.158 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.159 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642989317000001, 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.160 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.161 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:39.162 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.163 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642966436999998, 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.164 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.165 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:39.166 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.167 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.168 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642948373999999, 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.169 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:39.170 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:39.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.172 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.173 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642910886999999, 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:39.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.176 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:39.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.178 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642885264999997, 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.179 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.181 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.182 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:39.183 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.184 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642864416999998, 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.185 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.186 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.187 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:39.188 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.189 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.190 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642847177999997, 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.191 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.192 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.193 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:39.194 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642795083999999, 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.195 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:39.196 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.197 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.198 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:39.199 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642775211, 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.200 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.201 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.202 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:39.203 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.204 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642728963000003, 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.205 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.206 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.207 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.208 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642708276, 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:39.209 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:39.210 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.211 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:39.212 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.213 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642659586999997, 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.215 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.216 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:39.217 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.218 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642590534, 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.219 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:39.220 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.221 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:39.222 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.6425707, 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.223 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.224 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.225 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:39.226 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.227 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642522069000002, 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.228 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.229 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:39.230 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.231 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.232 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642501009, 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.233 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:39.234 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.235 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:39.236 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.237 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642450445999998, 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.238 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:39.240 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.241 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642429518999997, 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:39.242 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:39.243 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.244 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:39.245 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.246 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 52.642401632000002, 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:39.247 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.248 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:39.249 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:39.250 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.251 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:39.252 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:39.252 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:39.253 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:39.254 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:39.254 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:39.255 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:39.251+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:39.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:39.255 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:39.256 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:39.270 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:39.271 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:39.293 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:39.314 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:39.314 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:39.335 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:39.335 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:39.354 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:39.351+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:39.356 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:39.356 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:39.373 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:39.373 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:39.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:39.476 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:39.475+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:39.527 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:16:39.555 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:39.551+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:39.571 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:39.571 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:39.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:39.656 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:39.651+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:39.672 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:39.672 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:39.672 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-22T01:16:39.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-22T01:16:39.754 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:39.754 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:39.757 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:39.751+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:39.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:39.858 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:39.855+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:39.889 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:39.889 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:39.903 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:39.976 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:16:39.976 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 159, 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "fsid": "e212681a-0600-4f32-ab96-8ca46d37e9a4", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "created": "2024-09-22T01:13:34.805751+0000", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "modified": "2024-09-22T01:16:39.524193+0000", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "last_up_change": "2024-09-22T01:13:47.939773+0000", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "last_in_change": "2024-09-22T01:14:37.510453+0000", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_num": 5801984, 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_set": [ 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "noscrub", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "pglog_hardlimit", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "purged_snapdirs", 2024-09-22T01:16:39.977 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_deletes", 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "sortbitwise" 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_version": 5, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "full_ratio": 0.94999998807907104, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_snapshot": "", 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_max": 3, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd": 8, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "require_min_compat_client": "luminous", 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "min_compat_client": "luminous", 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "require_osd_release": "squid", 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_crimson": false, 2024-09-22T01:16:39.978 INFO:teuthology.orchestra.run.smithi096.stdout: "pools": [ 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 1, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": ".mgr", 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:49.594720+0000", 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool", 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:39.979 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 1, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 1, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 1, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 1, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 1, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:39.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "19", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 0, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 0, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:39.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:39.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "options": { 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_max": 32, 2024-09-22T01:16:39.983 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_min": 1 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": {} 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 8, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 8, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.25, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 2, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 2, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:39.984 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 2, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "rbd", 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:13:53.081727+0000", 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:39.985 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 8, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 8, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 8, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 8, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 8, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:39.986 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "23", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 2, 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 23, 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:39.987 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:39.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:39.989 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 5, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 5, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 0.75, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 3.75, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 3.75, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": "pool_client.0", 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "create_time": "2024-09-22T01:14:03.263074+0000", 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 8193, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": 1, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 2, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": 1, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_rule": 0, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_count": 0, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_target": 0, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_barrier": 0, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "is_stretch_pool": false, 2024-09-22T01:16:39.991 INFO:teuthology.orchestra.run.smithi096.stdout: "object_hash": 2, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscale_mode": "off", 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": 32, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num": 32, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_placement_num_target": 32, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_target": 32, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num_pending": 32, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "last_pg_merge_meta": { 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "source_pgid": "0.0", 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "ready_epoch": 0, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_started": 0, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "last_epoch_clean": 0, 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "source_version": "0'0", 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: "target_version": "0'0" 2024-09-22T01:16:39.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "last_change": "159", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend": "0", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "last_force_op_resend_preluminous": "0", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "auid": 0, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_mode": "selfmanaged", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_seq": 54, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_epoch": 159, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_snaps": [], 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps": "[]", 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_bytes": 0, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "quota_max_objects": 0, 2024-09-22T01:16:39.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tiers": [], 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tier_of": -1, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "read_tier": -1, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "write_tier": -1, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_mode": "none", 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_bytes": 0, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "target_max_objects": 0, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_flush_age": 0, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "cache_min_evict_age": 0, 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profile": "", 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_params": { 2024-09-22T01:16:39.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "none" 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_period": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_count": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "use_gmt_hitset": true, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min_read_recency_for_promote": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "min_write_recency_for_promote": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_grade_decay_rate": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "hit_set_search_last_n": 0, 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "grade_table": [], 2024-09-22T01:16:39.995 INFO:teuthology.orchestra.run.smithi096.stdout: "stripe_width": 0, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "expected_num_objects": 0, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "fast_read": false, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "options": {}, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "application_metadata": { 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd": {} 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "read_balance": { 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "score_type": "Fair distribution", 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "score_acting": 1.75, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "score_stable": 1.75, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "optimal_score": 1, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_acting": 1.75, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "raw_score_stable": 1.75, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity_weighted": 0.92731666564941406, 2024-09-22T01:16:39.996 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "a0e12724-c6a8-43ce-aad2-d3043c2f23d9", 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:39.997 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 14, 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-09-22T01:16:39.998 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6802", 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:39.999 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6803", 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6806", 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6807", 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6804", 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6805", 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3838492416 2024-09-22T01:16:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6801/3838492416", 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6803/3838492416", 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6807/3838492416", 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6805/3838492416", 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "bd73805d-596e-40f0-ba54-69ca84175c51", 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.003 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6824", 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6825", 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.005 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6830", 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6831", 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1053715553 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6825/1053715553", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6827/1053715553", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6831/1053715553", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6829/1053715553", 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.008 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6", 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.009 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 128, 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6816", 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6817", 2024-09-22T01:16:40.010 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6818", 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6819", 2024-09-22T01:16:40.011 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6822", 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6823", 2024-09-22T01:16:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6820", 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6821", 2024-09-22T01:16:40.013 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 860290285 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6817/860290285", 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6819/860290285", 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6823/860290285", 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6821/860290285", 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "eed7a22f-aa64-4dad-9151-8fc05271275e", 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 58, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.015 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6808", 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6809", 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.016 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6810", 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6811", 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.017 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6814", 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6815", 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.018 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6812", 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6813", 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 740986778 2024-09-22T01:16:40.019 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.96:6809/740986778", 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.96:6811/740986778", 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.96:6815/740986778", 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.96:6813/740986778", 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:40.020 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "14ed2997-6237-4ca6-a17b-0c0ae87f0645", 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 0.4185333251953125, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 118, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6800", 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6801", 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6802", 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6803", 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6806", 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6807", 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6804", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6805", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 787691336 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6801/787691336", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6803/787691336", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6807/787691336", 2024-09-22T01:16:40.025 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6805/787691336", 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "6ecd111f-9cc2-45a6-92f3-67f73bacfdad", 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.026 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 113, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6824", 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6825", 2024-09-22T01:16:40.027 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6826", 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6827", 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6830", 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6831", 2024-09-22T01:16:40.029 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6828", 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6829", 2024-09-22T01:16:40.030 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 721780330 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6825/721780330", 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6827/721780330", 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6831/721780330", 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6829/721780330", 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.031 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "5032e618-100c-4318-9825-3e90a3c4b25e", 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 1, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 112, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6816", 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6817", 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6818", 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6819", 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6822", 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6823", 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6820", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6821", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2103163729 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6817/2103163729", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6819/2103163729", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6823/2103163729", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6821/2103163729", 2024-09-22T01:16:40.036 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "uuid": "020cdc04-e2da-4af5-b6d9-674e88425186", 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "up": 1, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "in": 1, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "weight": 0.2866363525390625, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_affinity": 1, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_begin": 0, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "last_clean_end": 0, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "up_from": 15, 2024-09-22T01:16:40.037 INFO:teuthology.orchestra.run.smithi096.stdout: "up_thru": 95, 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "down_at": 0, 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "lost_at": 0, 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addrs": { 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6808", 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6809", 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addrs": { 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6810", 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6811", 2024-09-22T01:16:40.039 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addrs": { 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6814", 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6815", 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addrs": { 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6812", 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.150:6813", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1517191652 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "public_addr": "172.21.15.150:6809/1517191652", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_addr": "172.21.15.150:6811/1517191652", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_back_addr": "172.21.15.150:6815/1517191652", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "heartbeat_front_addr": "172.21.15.150:6813/1517191652", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "state": [ 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "exists", 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: "up" 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_xinfo": [ 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 0, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.358516+0000", 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.043 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 1, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.767062+0000", 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 2, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.044 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.745538+0000", 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 3, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.615624+0000", 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.045 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 4, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.063897+0000", 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 5, 2024-09-22T01:16:40.046 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.047 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.047 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.047 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.047 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.205999+0000", 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 6, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:44.352769+0000", 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": 7, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "down_stamp": "0.000000", 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_probability": 0, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "laggy_interval": 0, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "features": 4540701547738038271, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "old_weight": 0, 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "last_purged_snaps_scrub": "2024-09-22T01:13:43.943009+0000", 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "dead_epoch": 0 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap": [ 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.049 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.1", 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: 4, 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: 7 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: "pgid": "3.16", 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: "osds": [ 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: 2, 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: 0 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.050 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_items": [], 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_upmap_primaries": [], 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_temp": [], 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "primary_temp": [], 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklist": {}, 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "range_blocklist": {}, 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "erasure_code_profiles": { 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "default": { 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-failure-domain": "osd", 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "k": "2", 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "m": "1", 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "plugin": "jerasure", 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: "technique": "reed_sol_van" 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "removed_snaps_queue": [ 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 16, 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 8 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "new_removed_snaps": [ 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.052 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 16, 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 1 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "new_purged_snaps": [ 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "pool": 3, 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "snaps": [ 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:40.053 INFO:teuthology.orchestra.run.smithi096.stdout: "begin": 24, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "length": 4 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_node_flags": {}, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "device_class_flags": {}, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode": { 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_enabled": false, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_bucket_count": 0, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "degraded_stretch_mode": 0, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "recovering_stretch_mode": 0, 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: "stretch_mode_bucket": 0 2024-09-22T01:16:40.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:40.055 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:40.055 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:39.975+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:40.056 INFO:tasks.thrashosds.thrasher:j is {'epoch': 159, 'fsid': 'e212681a-0600-4f32-ab96-8ca46d37e9a4', 'created': '2024-09-22T01:13:34.805751+0000', 'modified': '2024-09-22T01:16:39.524193+0000', 'last_up_change': '2024-09-22T01:13:47.939773+0000', 'last_in_change': '2024-09-22T01:14:37.510453+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-22T01:13:49.594720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-22T01:13:53.081727+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-22T01:14:03.263074+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 54, 'snap_epoch': 159, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9273166656494141, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': 'a0e12724-c6a8-43ce-aad2-d3043c2f23d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 3838492416}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 3838492416}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 3838492416}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 3838492416}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 3838492416}]}, 'public_addr': '172.21.15.96:6801/3838492416', 'cluster_addr': '172.21.15.96:6803/3838492416', 'heartbeat_back_addr': '172.21.15.96:6807/3838492416', 'heartbeat_front_addr': '172.21.15.96:6805/3838492416', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'bd73805d-596e-40f0-ba54-69ca84175c51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6824', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6825', 'nonce': 1053715553}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6826', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6827', 'nonce': 1053715553}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6830', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6831', 'nonce': 1053715553}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1053715553}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1053715553}]}, 'public_addr': '172.21.15.96:6825/1053715553', 'cluster_addr': '172.21.15.96:6827/1053715553', 'heartbeat_back_addr': '172.21.15.96:6831/1053715553', 'heartbeat_front_addr': '172.21.15.96:6829/1053715553', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 860290285}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 860290285}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 860290285}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 860290285}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 860290285}]}, 'public_addr': '172.21.15.96:6817/860290285', 'cluster_addr': '172.21.15.96:6819/860290285', 'heartbeat_back_addr': '172.21.15.96:6823/860290285', 'heartbeat_front_addr': '172.21.15.96:6821/860290285', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'eed7a22f-aa64-4dad-9151-8fc05271275e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 740986778}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 740986778}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 740986778}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 740986778}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 740986778}]}, 'public_addr': '172.21.15.96:6809/740986778', 'cluster_addr': '172.21.15.96:6811/740986778', 'heartbeat_back_addr': '172.21.15.96:6815/740986778', 'heartbeat_front_addr': '172.21.15.96:6813/740986778', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '14ed2997-6237-4ca6-a17b-0c0ae87f0645', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4185333251953125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6800', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6801', 'nonce': 787691336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6802', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6803', 'nonce': 787691336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6806', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6807', 'nonce': 787691336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6804', 'nonce': 787691336}, {'type': 'v1', 'addr': '172.21.15.150:6805', 'nonce': 787691336}]}, 'public_addr': '172.21.15.150:6801/787691336', 'cluster_addr': '172.21.15.150:6803/787691336', 'heartbeat_back_addr': '172.21.15.150:6807/787691336', 'heartbeat_front_addr': '172.21.15.150:6805/787691336', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6ecd111f-9cc2-45a6-92f3-67f73bacfdad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6824', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6825', 'nonce': 721780330}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6826', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6827', 'nonce': 721780330}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6830', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6831', 'nonce': 721780330}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6828', 'nonce': 721780330}, {'type': 'v1', 'addr': '172.21.15.150:6829', 'nonce': 721780330}]}, 'public_addr': '172.21.15.150:6825/721780330', 'cluster_addr': '172.21.15.150:6827/721780330', 'heartbeat_back_addr': '172.21.15.150:6831/721780330', 'heartbeat_front_addr': '172.21.15.150:6829/721780330', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5032e618-100c-4318-9825-3e90a3c4b25e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6816', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6817', 'nonce': 2103163729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6818', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6819', 'nonce': 2103163729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6822', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6823', 'nonce': 2103163729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6820', 'nonce': 2103163729}, {'type': 'v1', 'addr': '172.21.15.150:6821', 'nonce': 2103163729}]}, 'public_addr': '172.21.15.150:6817/2103163729', 'cluster_addr': '172.21.15.150:6819/2103163729', 'heartbeat_back_addr': '172.21.15.150:6823/2103163729', 'heartbeat_front_addr': '172.21.15.150:6821/2103163729', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '020cdc04-e2da-4af5-b6d9-674e88425186', 'up': 1, 'in': 1, 'weight': 0.2866363525390625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6808', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6809', 'nonce': 1517191652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6810', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6811', 'nonce': 1517191652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6814', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6815', 'nonce': 1517191652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6812', 'nonce': 1517191652}, {'type': 'v1', 'addr': '172.21.15.150:6813', 'nonce': 1517191652}]}, 'public_addr': '172.21.15.150:6809/1517191652', 'cluster_addr': '172.21.15.150:6811/1517191652', 'heartbeat_back_addr': '172.21.15.150:6815/1517191652', 'heartbeat_front_addr': '172.21.15.150:6813/1517191652', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.358516+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.767062+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.745538+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.615624+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.063897+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.205999+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:44.352769+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-22T01:13:43.943009+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [4, 7]}, {'pgid': '3.16', 'osds': [2, 0]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 8}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 16, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 4}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-22T01:16:40.056 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:16:40.067 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.067 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.067 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:40.063+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:40.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:40.168 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:40.167+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:40.240 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.241 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:40.260 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:40.259+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:40.361 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:40.359+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:40.371 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:16:40.371 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:16:40.391 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2024-09-22T01:16:39.572213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508494949,"num_objects":875,"num_object_clones":92,"num_object_copies":1750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":875,"num_whiteouts":556,"num_read":267210,"num_read_kb":967335,"num_write":23211,"num_write_kb":379800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1277,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21008,"ondisk_log_size":21008,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":565668,"kb_used_data":170388,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754409052,"statfs":{"total":773094113280,"available":772514869248,"internally_reserved":0,"allocated":174477312,"data_stored":280936930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":404540976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-233707261,"num_objects":-139,"num_object_clones":-50,"num_object_copies":-278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-139,"num_whiteouts":-38,"num_read":2453,"num_read_kb":2079,"num_write":244,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001040"},"pg_stats":[{"pgid":"3.1f","version":"155'335","reported_seq":3168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.583236+0000","last_change":"2024-09-22T01:16:38.584136+0000","last_active":"2024-09-22T01:16:38.583236+0000","last_peered":"2024-09-22T01:16:38.583236+0000","last_clean":"2024-09-22T01:16:38.583236+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:16:38.583236+0000","last_undegraded":"2024-09-22T01:16:38.583236+0000","last_fullsized":"2024-09-22T01:16:38.583236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:18:21.004378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056296,"stat_sum":{"num_bytes":7217252,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":2470,"num_read_kb":21989,"num_write":290,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.1e","version":"157'537","reported_seq":12678,"reported_epoch":158,"state":"active+clean+snaptrim","last_fresh":"2024-09-22T01:16:38.583384+0000","last_change":"2024-09-22T01:16:38.583419+0000","last_active":"2024-09-22T01:16:38.583384+0000","last_peered":"2024-09-22T01:16:38.583384+0000","last_clean":"2024-09-22T01:16:38.583384+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:16:38.583384+0000","last_undegraded":"2024-09-22T01:16:38.583384+0000","last_fullsized":"2024-09-22T01:16:38.583384+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:55.724646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034528409999999999,"stat_sum":{"num_bytes":12176041,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":12573,"num_read_kb":37134,"num_write":538,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1d","version":"151'468","reported_seq":20398,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.573815+0000","last_change":"2024-09-22T01:16:38.574743+0000","last_active":"2024-09-22T01:16:38.573815+0000","last_peered":"2024-09-22T01:16:38.573815+0000","last_clean":"2024-09-22T01:16:38.573815+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:16:38.573815+0000","last_undegraded":"2024-09-22T01:16:38.573815+0000","last_fullsized":"2024-09-22T01:16:38.573815+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:18:06.129048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051854199999999996,"stat_sum":{"num_bytes":9932003,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":21023,"num_read_kb":28384,"num_write":416,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.1c","version":"157'655","reported_seq":12784,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.606366+0000","last_change":"2024-09-22T01:16:38.560422+0000","last_active":"2024-09-22T01:16:38.606366+0000","last_peered":"2024-09-22T01:16:38.606366+0000","last_clean":"2024-09-22T01:16:38.606366+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:16:38.606366+0000","last_undegraded":"2024-09-22T01:16:38.606366+0000","last_fullsized":"2024-09-22T01:16:38.606366+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:47:48.213740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025545059999999998,"stat_sum":{"num_bytes":25321525,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":17,"num_read":13222,"num_read_kb":38953,"num_write":747,"num_write_kb":12887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.1b","version":"155'393","reported_seq":5658,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.555992+0000","last_change":"2024-09-22T01:16:37.556897+0000","last_active":"2024-09-22T01:16:37.555992+0000","last_peered":"2024-09-22T01:16:37.555992+0000","last_clean":"2024-09-22T01:16:37.555992+0000","last_became_active":"2024-09-22T01:15:32.940539+0000","last_became_peered":"2024-09-22T01:15:32.940539+0000","last_unstale":"2024-09-22T01:16:37.555992+0000","last_undegraded":"2024-09-22T01:16:37.555992+0000","last_fullsized":"2024-09-22T01:16:37.555992+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:37:00.853880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019651500000000001,"stat_sum":{"num_bytes":565285,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":18,"num_read":5174,"num_read_kb":11303,"num_write":306,"num_write_kb":4889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.1a","version":"157'1157","reported_seq":17130,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.611078+0000","last_change":"2024-09-22T01:16:38.560470+0000","last_active":"2024-09-22T01:16:38.611078+0000","last_peered":"2024-09-22T01:16:38.611078+0000","last_clean":"2024-09-22T01:16:38.611078+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:16:38.611078+0000","last_undegraded":"2024-09-22T01:16:38.611078+0000","last_fullsized":"2024-09-22T01:16:38.611078+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:50:33.810133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023301239999999998,"stat_sum":{"num_bytes":20580669,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":20042,"num_read_kb":77815,"num_write":1693,"num_write_kb":27239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.19","version":"157'479","reported_seq":6340,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.626218+0000","last_change":"2024-09-22T01:16:37.626218+0000","last_active":"2024-09-22T01:16:37.626218+0000","last_peered":"2024-09-22T01:16:37.626218+0000","last_clean":"2024-09-22T01:16:37.626218+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:16:37.626218+0000","last_undegraded":"2024-09-22T01:16:37.626218+0000","last_fullsized":"2024-09-22T01:16:37.626218+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:09:57.607910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575324,"stat_sum":{"num_bytes":8987238,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":7146,"num_read_kb":28181,"num_write":363,"num_write_kb":6650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.18","version":"155'384","reported_seq":6485,"reported_epoch":158,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-22T01:16:38.583655+0000","last_change":"2024-09-22T01:16:38.583635+0000","last_active":"2024-09-22T01:16:38.583655+0000","last_peered":"2024-09-22T01:16:38.583655+0000","last_clean":"2024-09-22T01:16:38.583655+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:16:38.583655+0000","last_undegraded":"2024-09-22T01:16:38.583655+0000","last_fullsized":"2024-09-22T01:16:38.583655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:59:22.037262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037026120000000001,"stat_sum":{"num_bytes":17067805,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":6047,"num_read_kb":39370,"num_write":341,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.17","version":"157'550","reported_seq":5634,"reported_epoch":158,"state":"active+clean+snaptrim","last_fresh":"2024-09-22T01:16:38.584289+0000","last_change":"2024-09-22T01:16:38.584378+0000","last_active":"2024-09-22T01:16:38.584289+0000","last_peered":"2024-09-22T01:16:38.584289+0000","last_clean":"2024-09-22T01:16:38.584289+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:16:38.584289+0000","last_undegraded":"2024-09-22T01:16:38.584289+0000","last_fullsized":"2024-09-22T01:16:38.584289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:56:12.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034969210000000001,"stat_sum":{"num_bytes":18648863,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":6487,"num_read_kb":34442,"num_write":511,"num_write_kb":11881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.16","version":"156'497","reported_seq":13330,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.562953+0000","last_change":"2024-09-22T01:16:38.564834+0000","last_active":"2024-09-22T01:16:38.562953+0000","last_peered":"2024-09-22T01:16:38.562953+0000","last_clean":"2024-09-22T01:16:38.562953+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:16:38.562953+0000","last_undegraded":"2024-09-22T01:16:38.562953+0000","last_fullsized":"2024-09-22T01:16:38.562953+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:20:26.116023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024950749999999998,"stat_sum":{"num_bytes":11911486,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":29,"num_read":12253,"num_read_kb":27168,"num_write":485,"num_write_kb":13004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.15","version":"155'827","reported_seq":3946,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.558981+0000","last_change":"2024-09-22T01:16:37.560040+0000","last_active":"2024-09-22T01:16:37.558981+0000","last_peered":"2024-09-22T01:16:37.558981+0000","last_clean":"2024-09-22T01:16:37.558981+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:16:37.558981+0000","last_undegraded":"2024-09-22T01:16:37.558981+0000","last_fullsized":"2024-09-22T01:16:37.558981+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:01:27.344285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004756466,"stat_sum":{"num_bytes":4693314,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":13,"num_read":5678,"num_read_kb":18792,"num_write":1170,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.14","version":"157'718","reported_seq":10389,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.573369+0000","last_change":"2024-09-22T01:16:38.574611+0000","last_active":"2024-09-22T01:16:38.573369+0000","last_peered":"2024-09-22T01:16:38.573369+0000","last_clean":"2024-09-22T01:16:38.573369+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:16:38.573369+0000","last_undegraded":"2024-09-22T01:16:38.573369+0000","last_fullsized":"2024-09-22T01:16:38.573369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:26.255902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057096229999999996,"stat_sum":{"num_bytes":12171780,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":22,"num_read":12123,"num_read_kb":35522,"num_write":897,"num_write_kb":12773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":268,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.568611+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:16:38.568611+0000","last_peered":"2024-09-22T01:16:38.568611+0000","last_clean":"2024-09-22T01:16:38.568611+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:16:38.568611+0000","last_undegraded":"2024-09-22T01:16:38.568611+0000","last_fullsized":"2024-09-22T01:16:38.568611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"157'581","reported_seq":4324,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.629465+0000","last_change":"2024-09-22T01:16:37.629465+0000","last_active":"2024-09-22T01:16:37.629465+0000","last_peered":"2024-09-22T01:16:37.629465+0000","last_clean":"2024-09-22T01:16:37.629465+0000","last_became_active":"2024-09-22T01:15:59.611357+0000","last_became_peered":"2024-09-22T01:15:59.611357+0000","last_unstale":"2024-09-22T01:16:37.629465+0000","last_undegraded":"2024-09-22T01:16:37.629465+0000","last_fullsized":"2024-09-22T01:16:37.629465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:07:17.282633+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0035589440000000001,"stat_sum":{"num_bytes":16978031,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":16,"num_read":6327,"num_read_kb":41048,"num_write":516,"num_write_kb":23285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":115,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.625662+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:16:37.625662+0000","last_peered":"2024-09-22T01:16:37.625662+0000","last_clean":"2024-09-22T01:16:37.625662+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:16:37.625662+0000","last_undegraded":"2024-09-22T01:16:37.625662+0000","last_fullsized":"2024-09-22T01:16:37.625662+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"155'368","reported_seq":4213,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.570603+0000","last_change":"2024-09-22T01:16:38.571536+0000","last_active":"2024-09-22T01:16:38.570603+0000","last_peered":"2024-09-22T01:16:38.570603+0000","last_clean":"2024-09-22T01:16:38.570603+0000","last_became_active":"2024-09-22T01:15:32.926099+0000","last_became_peered":"2024-09-22T01:15:32.926099+0000","last_unstale":"2024-09-22T01:16:38.570603+0000","last_undegraded":"2024-09-22T01:16:38.570603+0000","last_fullsized":"2024-09-22T01:16:38.570603+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:49:45.271998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024779509999999999,"stat_sum":{"num_bytes":8899366,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":4224,"num_read_kb":26520,"num_write":294,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.5","version":"0'0","reported_seq":268,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.570053+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:16:38.570053+0000","last_peered":"2024-09-22T01:16:38.570053+0000","last_clean":"2024-09-22T01:16:38.570053+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:16:38.570053+0000","last_undegraded":"2024-09-22T01:16:38.570053+0000","last_fullsized":"2024-09-22T01:16:38.570053+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"157'637","reported_seq":3429,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.569449+0000","last_change":"2024-09-22T01:16:38.570177+0000","last_active":"2024-09-22T01:16:38.569449+0000","last_peered":"2024-09-22T01:16:38.569449+0000","last_clean":"2024-09-22T01:16:38.569449+0000","last_became_active":"2024-09-22T01:14:04.528219+0000","last_became_peered":"2024-09-22T01:14:04.528219+0000","last_unstale":"2024-09-22T01:16:38.569449+0000","last_undegraded":"2024-09-22T01:16:38.569449+0000","last_fullsized":"2024-09-22T01:16:38.569449+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:55:57.357278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001239939,"stat_sum":{"num_bytes":3292057,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":5354,"num_read_kb":19361,"num_write":757,"num_write_kb":6453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.4","version":"0'0","reported_seq":288,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.584060+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:16:38.584060+0000","last_peered":"2024-09-22T01:16:38.584060+0000","last_clean":"2024-09-22T01:16:38.584060+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:16:38.584060+0000","last_undegraded":"2024-09-22T01:16:38.584060+0000","last_fullsized":"2024-09-22T01:16:38.584060+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"158'1150","reported_seq":8605,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.569953+0000","last_change":"2024-09-22T01:16:38.570770+0000","last_active":"2024-09-22T01:16:38.569953+0000","last_peered":"2024-09-22T01:16:38.569953+0000","last_clean":"2024-09-22T01:16:38.569953+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:16:38.569953+0000","last_undegraded":"2024-09-22T01:16:38.569953+0000","last_fullsized":"2024-09-22T01:16:38.569953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1150,"log_dups_size":0,"ondisk_log_size":1150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:30:36.165381+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0090857490000000006,"stat_sum":{"num_bytes":6594592,"num_objects":27,"num_object_clones":1,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":21,"num_read":11576,"num_read_kb":39742,"num_write":1715,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.3","version":"156'886","reported_seq":6690,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.560314+0000","last_change":"2024-09-22T01:16:37.561611+0000","last_active":"2024-09-22T01:16:37.560314+0000","last_peered":"2024-09-22T01:16:37.560314+0000","last_clean":"2024-09-22T01:16:37.560314+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:16:37.560314+0000","last_undegraded":"2024-09-22T01:16:37.560314+0000","last_fullsized":"2024-09-22T01:16:37.560314+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:11.229533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005823674,"stat_sum":{"num_bytes":9669987,"num_objects":30,"num_object_clones":3,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":17,"num_read":7389,"num_read_kb":29757,"num_write":928,"num_write_kb":11388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.2","version":"23'2","reported_seq":272,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.569344+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:16:38.569344+0000","last_peered":"2024-09-22T01:16:38.569344+0000","last_clean":"2024-09-22T01:16:38.569344+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:16:38.569344+0000","last_undegraded":"2024-09-22T01:16:38.569344+0000","last_fullsized":"2024-09-22T01:16:38.569344+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"157'494","reported_seq":4154,"reported_epoch":158,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-22T01:16:38.583573+0000","last_change":"2024-09-22T01:16:38.583513+0000","last_active":"2024-09-22T01:16:38.583573+0000","last_peered":"2024-09-22T01:16:38.583573+0000","last_clean":"2024-09-22T01:16:38.583573+0000","last_became_active":"2024-09-22T01:15:51.529181+0000","last_became_peered":"2024-09-22T01:15:51.529181+0000","last_unstale":"2024-09-22T01:16:38.583573+0000","last_undegraded":"2024-09-22T01:16:38.583573+0000","last_fullsized":"2024-09-22T01:16:38.583573+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:08:57.794575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037312869999999998,"stat_sum":{"num_bytes":9290597,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":23,"num_read":4358,"num_read_kb":28278,"num_write":492,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.582903+0000","last_change":"2024-09-22T01:15:48.063999+0000","last_active":"2024-09-22T01:16:38.582903+0000","last_peered":"2024-09-22T01:16:38.582903+0000","last_clean":"2024-09-22T01:16:38.582903+0000","last_became_active":"2024-09-22T01:15:48.063387+0000","last_became_peered":"2024-09-22T01:15:48.063387+0000","last_unstale":"2024-09-22T01:16:38.582903+0000","last_undegraded":"2024-09-22T01:16:38.582903+0000","last_fullsized":"2024-09-22T01:16:38.582903+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:03:32.378683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"154'538","reported_seq":6807,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.557852+0000","last_change":"2024-09-22T01:16:38.558431+0000","last_active":"2024-09-22T01:16:38.557852+0000","last_peered":"2024-09-22T01:16:38.557852+0000","last_clean":"2024-09-22T01:16:38.557852+0000","last_became_active":"2024-09-22T01:15:32.937217+0000","last_became_peered":"2024-09-22T01:15:32.937217+0000","last_unstale":"2024-09-22T01:16:38.557852+0000","last_undegraded":"2024-09-22T01:16:38.557852+0000","last_fullsized":"2024-09-22T01:16:38.557852+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:30:22.482276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081492100000000003,"stat_sum":{"num_bytes":16124937,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":7960,"num_read_kb":17095,"num_write":487,"num_write_kb":11616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"2.0","version":"0'0","reported_seq":268,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.568257+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:16:38.568257+0000","last_peered":"2024-09-22T01:16:38.568257+0000","last_clean":"2024-09-22T01:16:38.568257+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:16:38.568257+0000","last_undegraded":"2024-09-22T01:16:38.568257+0000","last_fullsized":"2024-09-22T01:16:38.568257+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"157'636","reported_seq":6287,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.746861+0000","last_change":"2024-09-22T01:16:37.628905+0000","last_active":"2024-09-22T01:16:37.746861+0000","last_peered":"2024-09-22T01:16:37.746861+0000","last_clean":"2024-09-22T01:16:37.746861+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:16:37.746861+0000","last_undegraded":"2024-09-22T01:16:37.746861+0000","last_fullsized":"2024-09-22T01:16:37.746861+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:12:50.220210+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0032318220000000001,"stat_sum":{"num_bytes":29409312,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":23,"num_read":8044,"num_read_kb":25761,"num_write":685,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"2.3","version":"21'1","reported_seq":290,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.569010+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:16:38.569010+0000","last_peered":"2024-09-22T01:16:38.569010+0000","last_clean":"2024-09-22T01:16:38.569010+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:16:38.569010+0000","last_undegraded":"2024-09-22T01:16:38.569010+0000","last_fullsized":"2024-09-22T01:16:38.569010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":440,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.558175+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:16:38.558175+0000","last_peered":"2024-09-22T01:16:38.558175+0000","last_clean":"2024-09-22T01:16:38.558175+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:16:38.558175+0000","last_undegraded":"2024-09-22T01:16:38.558175+0000","last_fullsized":"2024-09-22T01:16:38.558175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"155'470","reported_seq":3769,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.561807+0000","last_change":"2024-09-22T01:16:38.562795+0000","last_active":"2024-09-22T01:16:38.561807+0000","last_peered":"2024-09-22T01:16:38.561807+0000","last_clean":"2024-09-22T01:16:38.561807+0000","last_became_active":"2024-09-22T01:15:51.530462+0000","last_became_peered":"2024-09-22T01:15:51.530462+0000","last_unstale":"2024-09-22T01:16:38.561807+0000","last_undegraded":"2024-09-22T01:16:38.561807+0000","last_fullsized":"2024-09-22T01:16:38.561807+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:56.403199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014028039999999999,"stat_sum":{"num_bytes":48258631,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":3742,"num_read_kb":17323,"num_write":341,"num_write_kb":4201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.9","version":"156'955","reported_seq":19152,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.570351+0000","last_change":"2024-09-22T01:16:38.571578+0000","last_active":"2024-09-22T01:16:38.570351+0000","last_peered":"2024-09-22T01:16:38.570351+0000","last_clean":"2024-09-22T01:16:38.570351+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:16:38.570351+0000","last_undegraded":"2024-09-22T01:16:38.570351+0000","last_fullsized":"2024-09-22T01:16:38.570351+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:06.469716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00292706,"stat_sum":{"num_bytes":69777372,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":22,"num_read":24118,"num_read_kb":81058,"num_write":936,"num_write_kb":23103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.a","version":"157'648","reported_seq":4172,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.571829+0000","last_change":"2024-09-22T01:16:38.573875+0000","last_active":"2024-09-22T01:16:38.571829+0000","last_peered":"2024-09-22T01:16:38.571829+0000","last_clean":"2024-09-22T01:16:38.571829+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:16:38.571829+0000","last_undegraded":"2024-09-22T01:16:38.571829+0000","last_fullsized":"2024-09-22T01:16:38.571829+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:25:06.791217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047280819999999998,"stat_sum":{"num_bytes":12976143,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":13,"num_read":7376,"num_read_kb":11932,"num_write":607,"num_write_kb":8463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.b","version":"158'543","reported_seq":4788,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.577197+0000","last_change":"2024-09-22T01:16:38.577780+0000","last_active":"2024-09-22T01:16:38.577197+0000","last_peered":"2024-09-22T01:16:38.577197+0000","last_clean":"2024-09-22T01:16:38.577197+0000","last_became_active":"2024-09-22T01:15:50.529119+0000","last_became_peered":"2024-09-22T01:15:50.529119+0000","last_unstale":"2024-09-22T01:16:38.577197+0000","last_undegraded":"2024-09-22T01:16:38.577197+0000","last_fullsized":"2024-09-22T01:16:38.577197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:53.293662+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088039929999999995,"stat_sum":{"num_bytes":13976768,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":13,"num_read":5502,"num_read_kb":17997,"num_write":639,"num_write_kb":9237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.c","version":"156'676","reported_seq":5796,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.556151+0000","last_change":"2024-09-22T01:16:37.557016+0000","last_active":"2024-09-22T01:16:37.556151+0000","last_peered":"2024-09-22T01:16:37.556151+0000","last_clean":"2024-09-22T01:16:37.556151+0000","last_became_active":"2024-09-22T01:15:51.537372+0000","last_became_peered":"2024-09-22T01:15:51.537372+0000","last_unstale":"2024-09-22T01:16:37.556151+0000","last_undegraded":"2024-09-22T01:16:37.556151+0000","last_fullsized":"2024-09-22T01:16:37.556151+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:27.159294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020508810000000001,"stat_sum":{"num_bytes":21674952,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":5633,"num_read_kb":28845,"num_write":728,"num_write_kb":10039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.d","version":"150'680","reported_seq":2901,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.574886+0000","last_change":"2024-09-22T01:16:38.575927+0000","last_active":"2024-09-22T01:16:38.574886+0000","last_peered":"2024-09-22T01:16:38.574886+0000","last_clean":"2024-09-22T01:16:38.574886+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:16:38.574886+0000","last_undegraded":"2024-09-22T01:16:38.574886+0000","last_fullsized":"2024-09-22T01:16:38.574886+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:17:18.943345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0060221659999999998,"stat_sum":{"num_bytes":6059408,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":849,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.e","version":"156'865","reported_seq":3967,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.565133+0000","last_change":"2024-09-22T01:16:38.566195+0000","last_active":"2024-09-22T01:16:38.565133+0000","last_peered":"2024-09-22T01:16:38.565133+0000","last_clean":"2024-09-22T01:16:38.565133+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:16:38.565133+0000","last_undegraded":"2024-09-22T01:16:38.565133+0000","last_fullsized":"2024-09-22T01:16:38.565133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:27.316024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00325677,"stat_sum":{"num_bytes":5808961,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":5414,"num_read_kb":21454,"num_write":1083,"num_write_kb":11694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.f","version":"154'696","reported_seq":3931,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-22T01:16:37.558602+0000","last_change":"2024-09-22T01:16:37.559685+0000","last_active":"2024-09-22T01:16:37.558602+0000","last_peered":"2024-09-22T01:16:37.558602+0000","last_clean":"2024-09-22T01:16:37.558602+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:16:37.558602+0000","last_undegraded":"2024-09-22T01:16:37.558602+0000","last_fullsized":"2024-09-22T01:16:37.558602+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:09:27.055764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046011910000000001,"stat_sum":{"num_bytes":7995443,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":4956,"num_read_kb":23182,"num_write":810,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"a"}]},{"pgid":"3.10","version":"156'654","reported_seq":3705,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.558223+0000","last_change":"2024-09-22T01:16:38.559206+0000","last_active":"2024-09-22T01:16:38.558223+0000","last_peered":"2024-09-22T01:16:38.558223+0000","last_clean":"2024-09-22T01:16:38.558223+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:16:38.558223+0000","last_undegraded":"2024-09-22T01:16:38.558223+0000","last_fullsized":"2024-09-22T01:16:38.558223+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:49:36.903201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014013540000000001,"stat_sum":{"num_bytes":5427136,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":689,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.11","version":"156'903","reported_seq":5056,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.570245+0000","last_change":"2024-09-22T01:16:38.571506+0000","last_active":"2024-09-22T01:16:38.570245+0000","last_peered":"2024-09-22T01:16:38.570245+0000","last_clean":"2024-09-22T01:16:38.570245+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:16:38.570245+0000","last_undegraded":"2024-09-22T01:16:38.570245+0000","last_fullsized":"2024-09-22T01:16:38.570245+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:01:05.250239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029472330000000001,"stat_sum":{"num_bytes":16564165,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":7735,"num_read_kb":50917,"num_write":1085,"num_write_kb":22546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.12","version":"156'769","reported_seq":5075,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.569301+0000","last_change":"2024-09-22T01:16:38.570534+0000","last_active":"2024-09-22T01:16:38.569301+0000","last_peered":"2024-09-22T01:16:38.569301+0000","last_clean":"2024-09-22T01:16:38.569301+0000","last_became_active":"2024-09-22T01:14:10.544340+0000","last_became_peered":"2024-09-22T01:14:10.544340+0000","last_unstale":"2024-09-22T01:16:38.569301+0000","last_undegraded":"2024-09-22T01:16:38.569301+0000","last_fullsized":"2024-09-22T01:16:38.569301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:12:25.717247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014941889999999999,"stat_sum":{"num_bytes":16511763,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":6620,"num_read_kb":26683,"num_write":898,"num_write_kb":11322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"11","length":"b"}]},{"pgid":"3.13","version":"158'739","reported_seq":5811,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-22T01:16:38.599079+0000","last_change":"2024-09-22T01:16:38.573210+0000","last_active":"2024-09-22T01:16:38.599079+0000","last_peered":"2024-09-22T01:16:38.599079+0000","last_clean":"2024-09-22T01:16:38.599079+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:16:38.599079+0000","last_undegraded":"2024-09-22T01:16:38.599079+0000","last_fullsized":"2024-09-22T01:16:38.599079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:40:48.757221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045385750000000004,"stat_sum":{"num_bytes":33341680,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":21,"num_read":7623,"num_read_kb":33027,"num_write":754,"num_write_kb":7638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"11","length":"b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":507904562,"num_objects":871,"num_object_clones":92,"num_object_copies":1742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":871,"num_whiteouts":556,"num_read":267094,"num_read_kb":967235,"num_write":23050,"num_write_kb":377382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1272,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156082176,"data_stored":269947165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20888,"ondisk_log_size":20888,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43296,"kb_used_data":3780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328544,"statfs":{"total":96636764160,"available":96592429056,"internally_reserved":0,"allocated":3870720,"data_stored":4322703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57808,"kb_used_data":8572,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94314032,"statfs":{"total":96636764160,"available":96577568768,"internally_reserved":0,"allocated":8777728,"data_stored":12036985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509477,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73992,"kb_used_data":24760,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94297848,"statfs":{"total":96636764160,"available":96560996352,"internally_reserved":0,"allocated":25354240,"data_stored":42262322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58408,"kb_used_data":27220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94313432,"statfs":{"total":96636764160,"available":96576954368,"internally_reserved":0,"allocated":27873280,"data_stored":45500999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86052,"kb_used_data":28496,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94285788,"statfs":{"total":96636764160,"available":96548646912,"internally_reserved":0,"allocated":29179904,"data_stored":47205747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80004,"kb_used_data":22412,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94291836,"statfs":{"total":96636764160,"available":96554840064,"internally_reserved":0,"allocated":22949888,"data_stored":37240656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85320,"kb_used_data":31916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94286520,"statfs":{"total":96636764160,"available":96549396480,"internally_reserved":0,"allocated":32681984,"data_stored":54600102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80788,"kb_used_data":23232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94291052,"statfs":{"total":96636764160,"available":96554037248,"internally_reserved":0,"allocated":23789568,"data_stored":37767416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21811200,"data_stored":36276646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30208000,"data_stored":53110830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20344832,"data_stored":35621792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":26951680,"data_stored":45977800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25993216,"data_stored":44602095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":22605824,"data_stored":40502492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6176768,"data_stored":10431711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1990656,"data_stored":3423799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:16:40.393 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [6, 0, 7, 4] 2024-09-22T01:16:40.393 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.15', '6', '0', '7', '4'] 2024-09-22T01:16:40.394 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.15 6 0 7 4 2024-09-22T01:16:40.427 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.427 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:40.462 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:40.459+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:40.562 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:40.555+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:40.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.619 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.636 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:40.680 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:40.679+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:40.761 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.761 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.764 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:40.759+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:40.775 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:40.865 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:40.863+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:40.925 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:40.925 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:40.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:40.966 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:40.959+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:41.057 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:41.058 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:41.067 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:41.063+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:41.070 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:41.168 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:41.167+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:41.187 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:41.188 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:41.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:41.269 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:41.263+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:41.337 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:41.352 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:41.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:41.395 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:41.391+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:41.471 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:41.467+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:41.513 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:41.513 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:41.525 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:41.525 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:41.527 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:41.541 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:41.542 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:41.558 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:41.558 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:41.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-22T01:16:41.570 INFO:teuthology.orchestra.run.smithi096.stderr:set 3.15 pg_upmap_items mapping to [6->0,7->4] 2024-09-22T01:16:41.572 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:41.567+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:41.583 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:41.583 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:41.607 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:41.627 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:41.628 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:41.652 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:41.652 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:41.670 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:41.671 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:41.672 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:41.667+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:41.703 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 96.895984999000007, 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.704 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.705 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.706 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.707 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:41.708 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 88.688595660000004, 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:41.709 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:41.710 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:41.711 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.712 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015986841999997, 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.713 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.714 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:41.715 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:41.716 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.717 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015966712000001, 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.718 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:41.719 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.720 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.721 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015945881999997, 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.722 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.723 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.724 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:41.725 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.726 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015925856999999, 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.727 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:41.728 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:41.729 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:41.730 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015905177, 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.731 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.732 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.733 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.734 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015883031000001, 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.735 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.736 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:41.737 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:41.738 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015346418999997, 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:41.739 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.740 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.741 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:41.742 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.743 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015321769000003, 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.744 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.745 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.746 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.015285403999997, 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.748 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.749 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.750 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.012336036000001, 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.752 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:41.753 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.755 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010728071000003, 2024-09-22T01:16:41.756 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.757 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:41.758 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:41.759 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.760 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010707058999998, 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.761 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.762 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.763 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.764 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010678036999998, 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:41.765 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:41.766 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:41.767 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.768 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010645689, 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:41.769 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:41.770 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.771 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.772 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010579569000001, 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.773 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:41.774 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.775 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.776 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:41.777 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.010549161, 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:41.778 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.779 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.780 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.781 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.009052537000002, 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:41.782 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.783 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.784 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.785 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 42.009022176000002, 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:41.786 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:41.787 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:41.788 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:41.789 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:41.790 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:41.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:41.791 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:41.786+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:41.892 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:41.886+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:41.964 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:41.964 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:41.964 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:41.964 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:41.978 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:42.027 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:42.023+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:42.093 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:42.090+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:42.101 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:42.101 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:42.101 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:42.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:42.194 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:42.190+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.286 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.138023164000003, 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.287 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.288 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.289 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:42.290 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137923224000005, 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.291 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.292 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.293 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.294 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137854333000007, 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.295 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.296 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.297 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.298 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137778269999998, 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.299 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.300 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:42.301 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:42.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:42.303 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137715482000004, 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137084548000004, 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:42.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137055408999998, 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.311 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.137026083999999, 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:42.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136995447999993, 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136959779999998, 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:42.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:42.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:42.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136930797000005, 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136896585000002, 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:42.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:42.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:42.335 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136827765000007, 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136727250999996, 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136659097000006, 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:42.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:42.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136575367000006, 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.348 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:42.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:42.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136441489000006, 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.355 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136314639999995, 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.356 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.359 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136277684999996, 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:42.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 93.136226042999994, 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.367 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:42.369 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:42.369 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:42.366+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:42.470 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:42.466+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:42.494 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:42.494 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:42.494 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:42.494 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:42.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:42.571 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:42.566+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:42.592 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is set 2024-09-22T01:16:42.643 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:42.643 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:42.644 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:42.644 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:42.644 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:42.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:42.703 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:42.699+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:42.772 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:42.766+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568926714, 2024-09-22T01:16:42.789 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.790 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.791 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:42.792 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.793 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568896633999998, 2024-09-22T01:16:42.794 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:42.795 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.796 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.797 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:42.798 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568875403, 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.799 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:42.800 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:42.801 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.802 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.803 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568854139000003, 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:42.804 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:42.805 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.806 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.807 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568832831999998, 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.808 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.809 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.810 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.811 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.812 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568811191999998, 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.813 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.814 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.815 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:42.816 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568766062000002, 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.817 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.818 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.819 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.820 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.821 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568735037000003, 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:42.822 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.823 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:42.824 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.825 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.826 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568710998999997, 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.827 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.828 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:42.829 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:42.830 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568689735, 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.831 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.832 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:42.833 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:42.834 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568637703, 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:42.835 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.836 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.837 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.838 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.839 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568578731999999, 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.840 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.841 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.842 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:42.843 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.568474965, 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.844 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.845 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:42.846 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:42.847 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567895182999997, 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:42.848 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.849 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.850 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.851 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.852 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567774739999997, 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.853 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:42.854 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:42.855 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.856 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.56769474, 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:42.857 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:42.858 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:42.859 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:42.860 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567230551999998, 2024-09-22T01:16:42.861 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.862 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.863 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:42.864 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.865 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567160977, 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.866 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:42.867 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.868 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.869 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.870 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567094228000002, 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:42.871 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.872 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.873 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.874 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 50.567022610000002, 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:42.875 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:42.876 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:42.877 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:42.878 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:42.879 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:42.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:42.881 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:42.878+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:42.981 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:42.978+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:43.049 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:43.049 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:43.049 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:43.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:43.082 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:43.078+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:43.183 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:43.178+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:43.188 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:43.188 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:43.188 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:43.188 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:43.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:43.284 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:43.278+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.329 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818567401000003, 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:43.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:43.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.333 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818508770999998, 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:43.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818485891000002, 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.337 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:43.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:43.340 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818467828000003, 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:43.341 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:43.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:43.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818430341000003, 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:43.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:43.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818404719, 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.348 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818383871000002, 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.352 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.355 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818366632, 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:43.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:43.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818314538000003, 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.360 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.363 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818294665000003, 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:43.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818248417, 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:43.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818227729999997, 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.373 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.375 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818179041, 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:43.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818109988000003, 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:43.379 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:43.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818090153999997, 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.383 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:43.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.385 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.386 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818041522999998, 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.387 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:43.388 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.389 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.390 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.818020463000003, 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:43.392 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:43.393 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.394 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.817969900000001, 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.395 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.396 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.397 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.398 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.817948973, 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.400 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:43.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 56.817921085999998, 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.404 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:43.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:43.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.407 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.408 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:43.408 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:43.408 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:43.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:43.410 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:43.406+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:43.510 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:43.507+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:43.541 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:43.541 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:43.541 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:43.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:43.611 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:43.607+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:43.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:43.699 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:43.699 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:43.699 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:43.713 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:43.707+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:43.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:43.813 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:43.810+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:43.835 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:43.851 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:43.852 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:43.868 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:43.869 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:43.879 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:43.879 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:43.885 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:43.885 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:43.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:43.906 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:43.906 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:43.927 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:43.927 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:43.927 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:43.922+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:43.947 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:43.947 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:43.968 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:43.968 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:44.014 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:44.011+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:44.029 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.029 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.043 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:44.115 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:44.111+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:44.203 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.203 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.215 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:44.211+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:44.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:44.316 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:44.311+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:44.342 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.342 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:44.417 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:44.415+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:44.473 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.473 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:44.518 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:44.511+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:44.619 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:44.615+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:44.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.620 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:16:44.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:44.724 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:44.722+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:44.808 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.808 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.820 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:44.815+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:44.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:44.921 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:44.915+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:44.979 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:44.979 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:44.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:45.021 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:45.015+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:45.116 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.116 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.122 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:45.118+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:45.130 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:45.223 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:45.219+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:45.294 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.294 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:45.324 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:45.322+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:45.425 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:45.419+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:45.439 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.439 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.453 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:45.539 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:45.535+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:45.601 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:16:45.625 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.625 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.626 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:45.622+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:45.640 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:45.727 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:45.723+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:45.813 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.813 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:45.828 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:45.823+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:45.928 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:45.923+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:45.991 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:45.991 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:45.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:46.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:46.037 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:46.034+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:46.131 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:46.126+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:46.149 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:46.149 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:46.166 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:46.166 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:46.167 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:46.167 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:46.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:46.187 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:46.187 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:46.200 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:46.200 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:46.219 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:46.219 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:46.231 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:46.226+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:46.239 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:46.239 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:46.258 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:46.258 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:46.278 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:46.278 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.153622041000006, 2024-09-22T01:16:46.302 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.303 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.304 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.305 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:46.306 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.153522100999993, 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.307 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.308 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.309 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.310 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.153453209999995, 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.311 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:46.312 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:46.313 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.314 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.153377147, 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.315 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:46.316 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:46.317 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:46.318 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.153314359000007, 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.319 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.320 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.321 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.322 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152683425000006, 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:46.323 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:46.324 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:46.325 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.326 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152654286000001, 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.327 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.328 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.329 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.330 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152624961000001, 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.331 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.332 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:46.333 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.334 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152594324999995, 2024-09-22T01:16:46.335 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:46.336 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.337 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.339 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152558657, 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.340 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.341 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.341 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.342 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.343 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:46.344 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152529673999993, 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.345 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.346 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.347 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152495462000005, 2024-09-22T01:16:46.348 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.349 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.350 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:46.351 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152426641999995, 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152326127999999, 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.356 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:46.358 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.359 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152257973999994, 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.360 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.361 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.362 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:46.363 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152174243999994, 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.364 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.365 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:46.366 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.367 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.152040365999994, 2024-09-22T01:16:46.368 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.370 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.371 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.372 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.151913516999997, 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.373 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.374 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:46.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.376 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.151876561999998, 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.377 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.378 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.379 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.380 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 97.151824919999996, 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.381 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:46.382 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:46.383 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.384 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:46.386 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:46.386 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:46.382+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:46.487 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:46.483+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:46.511 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:46.511 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:46.511 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:46.511 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:46.525 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:46.608 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:46.608 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:46.608 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-22T01:16:46.608 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-22T01:16:46.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-22T01:16:46.611 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:46.607+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:46.688 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:46.683+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:46.704 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:46.704 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:46.704 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:46.704 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:46.704 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:46.719 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:46.789 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:46.786+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:46.889 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669391834000002, 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:46.890 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.891 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.892 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.893 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669361754000001, 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.894 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.895 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.896 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.897 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.898 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669340523000002, 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.899 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.900 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:46.901 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:46.902 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669319258999998, 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.903 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.904 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.905 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:46.906 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.907 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669297952000001, 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.908 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.909 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.910 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.911 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.912 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669276312000001, 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.913 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:46.916 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669231181999997, 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.917 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.918 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:46.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.920 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.921 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669200156999999, 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:46.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.923 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.924 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.925 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669176118999999, 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:46.926 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.927 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.928 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.929 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.930 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669154855000002, 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.931 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:46.932 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.933 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.934 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669102823000003, 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.935 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.936 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.937 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.938 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.669043852000001, 2024-09-22T01:16:46.939 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:46.941 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:46.942 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.943 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.668940085000003, 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.944 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.945 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.946 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:46.947 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.948 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.668360303, 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.949 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:46.950 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.951 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.952 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.66823986, 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:46.953 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.954 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.955 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.956 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.668159860000003, 2024-09-22T01:16:46.957 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.958 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.959 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.960 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.961 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.667695672000001, 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.962 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.963 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:46.964 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:46.965 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.667626097000003, 2024-09-22T01:16:46.966 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.967 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.968 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:46.969 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.667559347999997, 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:46.970 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.971 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:46.972 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:46.973 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.667487729999998, 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:46.974 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.975 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.976 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:46.977 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:46.978 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:46.979 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:46.980 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:46.981 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:46.974+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:47.081 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:47.078+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:47.108 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:47.108 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:47.109 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:47.122 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:47.182 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:47.175+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:47.256 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:47.283 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:47.278+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:47.384 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:47.379+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.916000077, 2024-09-22T01:16:47.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:47.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.431 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915941447000002, 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.432 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:47.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:47.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915918566999999, 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.436 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.437 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915900504, 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:47.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.443 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915863017, 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:47.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.447 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915837394999997, 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.448 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915816546999999, 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:47.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.455 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915799307999997, 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:47.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915747214, 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.459 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:47.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915727341, 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.463 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:47.466 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915681093000003, 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.467 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.470 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915660406000001, 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:47.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.474 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915611716999997, 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:47.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:47.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915542664, 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:47.478 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.479 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:47.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:47.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.91552283, 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.482 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.484 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915474199000002, 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.486 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:47.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.489 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915453139, 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:47.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:47.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:47.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.493 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915402575999998, 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.494 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:47.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.497 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915381648999997, 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:47.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:47.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:47.501 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 60.915353762000002, 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:47.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:47.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:47.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:47.505 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:47.506 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:47.507 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:47.503+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:47.608 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:47.602+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:47.616 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-22T01:16:47.637 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:209336 3.1b 3:dbf05676:::rbd_header.11241ea8864d:head [watch unwatch cookie 139655089723872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e167)", 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:16:47.649064+0000", 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0018901809999999999, 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001909006, 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 209336 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649064+0000", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649064+0000", 2024-09-22T01:16:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649065+0000", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6599999999999996e-07 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649069+0000", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8820000000000003e-06 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649079+0000", 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8940000000000005e-06 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649085+0000", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5699999999999998e-06 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649266+0000", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018060600000000001 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649346+0000", 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0758000000000002e-05 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649514+0000", 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016775000000000001 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:209337 3.1b 3:dbf05676:::rbd_header.11241ea8864d:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e167)", 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:16:47.649314+0000", 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.001640175, 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017098739999999999, 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for rw locks", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 209337 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649314+0000", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649314+0000", 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649315+0000", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6400000000000002e-07 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649319+0000", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2130000000000002e-06 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649328+0000", 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4229999999999994e-06 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649335+0000", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4740000000000002e-06 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.649968+0000", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063337799999999998 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for rw locks", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:16:47.650058+0000", 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9888000000000004e-05 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-22T01:16:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2024-09-22T01:16:47.659 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:47.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:47.709 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:47.703+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:47.810 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:47.806+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:47.843 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:47.843 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:47.843 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:47.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:47.944 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:47.942+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:48.012 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:48.007+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 76.067959071999994, 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.912109771000004, 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:48.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:48.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.912088441999998, 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.026 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.911625369000006, 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:48.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.032 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:48.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.911598498999993, 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:48.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.911574990999995, 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:48.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:48.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.911378045000006, 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:48.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:48.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:48.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.911349978999993, 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.910108539999996, 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.910070774000005, 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:48.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:48.062 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.910046820000005, 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:48.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.066 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.910025734000001, 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:48.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.910005269999999, 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909985249000002, 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:48.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909966050999998, 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909944938999999, 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909924347, 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:48.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909902986000006, 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:48.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:48.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:48.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909881671999997, 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:48.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 70.909862653999994, 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:48.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:48.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:48.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:48.105 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:48.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:48.148 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:48.146+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:48.213 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:48.211+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:48.268 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:48.268 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:48.268 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:48.268 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:48.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:48.297 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-22T01:16:48.314 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:48.311+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:48.395 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:48.395 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:48.396 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:48.396 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:48.396 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:48.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:48.415 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:48.411+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:48.461 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:48.461 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:48.478 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:48.478 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:48.494 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:48.494 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:48.511 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:48.511 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:48.515 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:48.511+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:48.519 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:48.519 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:48.531 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:48.531 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:48.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:48.551 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:48.551 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:48.572 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:48.572 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:48.592 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:48.592 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:48.640 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:48.640 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:48.638+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:48.658 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:48.658 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:48.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:48.716 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:48.714+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:48.818 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:48.811+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:48.837 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:48.837 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:48.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:48.919 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:48.914+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:48.960 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:48.960 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:48.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:49.020 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:49.015+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:49.122 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:49.119+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:49.123 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.124 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:49.222 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:49.218+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:49.255 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.256 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:49.361 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:49.358+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:49.424 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:49.422+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:49.434 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.434 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:49.525 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:49.519+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:49.647 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:49.643+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:49.727 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:49.722+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:49.778 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.778 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:49.827 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:49.822+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:49.916 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:49.917 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:49.928 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:49.923+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:49.930 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:50.029 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:50.026+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:50.101 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.101 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.115 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:50.130 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:50.126+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:50.231 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:50.226+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:50.236 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.236 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.252 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:50.332 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:50.326+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:50.369 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.369 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.384 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:50.432 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:50.427+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:50.500 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.501 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.515 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:50.533 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:50.530+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:50.609 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-22T01:16:50.656 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:50.654+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:50.662 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-22T01:16:50.692 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.692 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:50.735 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:50.730+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:50.778 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0: {} 2024-09-22T01:16:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-22T01:16:50.800 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1: {} 2024-09-22T01:16:50.801 INFO:teuthology.orchestra.run.smithi096.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-22T01:16:50.821 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2: {} 2024-09-22T01:16:50.821 INFO:teuthology.orchestra.run.smithi096.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-22T01:16:50.836 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:50.831+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:50.840 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3: {} 2024-09-22T01:16:50.840 INFO:teuthology.orchestra.run.smithi096.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-22T01:16:50.841 INFO:teuthology.orchestra.run.smithi150.stdout:ERROR: (22) Invalid argument 2024-09-22T01:16:50.841 INFO:teuthology.orchestra.run.smithi150.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-22T01:16:50.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:50.858 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4: {} 2024-09-22T01:16:50.859 INFO:teuthology.orchestra.run.smithi096.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-22T01:16:50.877 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5: {} 2024-09-22T01:16:50.877 INFO:teuthology.orchestra.run.smithi096.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-22T01:16:50.897 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6: {} 2024-09-22T01:16:50.897 INFO:teuthology.orchestra.run.smithi096.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-22T01:16:50.916 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7: {} 2024-09-22T01:16:50.916 INFO:teuthology.orchestra.run.smithi096.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-22T01:16:50.976 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:50.974+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:51.024 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:51.024 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:51.024 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:51.024 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:51.037 INFO:tasks.ceph.osd.4.smithi150.stderr:2024-09-22T01:16:51.031+0000 7f9aca19a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26567) UID: 0 2024-09-22T01:16:51.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:51.138 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:51.131+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:51.212 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:51.212 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:51.212 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:51.212 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:51.212 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:51.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:51.239 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:51.238+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:51.340 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:51.335+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.399 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888286229000002, 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.400 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.401 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:51.402 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:51.403 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.404 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888227599000004, 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.405 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.406 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:51.407 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.408 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888204719000001, 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.409 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.410 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.411 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:51.412 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.413 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888186656000002, 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.414 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:51.415 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.416 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:51.417 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.418 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888149169000002, 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.419 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.420 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:51.421 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.422 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888123547000006, 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:51.423 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:51.424 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.425 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:51.426 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.427 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888102699000001, 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.428 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.429 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.430 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.431 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.432 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888085459999999, 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.433 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.434 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.435 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.436 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888033366000002, 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.437 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.438 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.441 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.888013493000003, 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.442 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:51.443 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.444 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:51.445 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.446 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887967244999999, 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.447 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:51.448 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.449 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:51.450 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887946557999996, 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.451 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.452 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:51.453 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.454 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.455 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887897869, 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.457 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:51.458 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.459 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887828815999995, 2024-09-22T01:16:51.460 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:51.462 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.463 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.464 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887808981999996, 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:51.465 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.466 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.467 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.468 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.469 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887760350999997, 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:51.470 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.471 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:51.472 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.473 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887739291000003, 2024-09-22T01:16:51.474 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:51.475 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.476 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:51.477 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.478 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887688728000001, 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.479 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.480 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.481 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887667801000006, 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.484 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:51.485 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.486 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.487 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 64.887639914000005, 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:51.488 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.489 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.490 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:51.491 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.492 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.493 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:51.493 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:51.493 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:51.494 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-22T01:16:51.495 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-22T01:16:51.490+0000 7fe439723640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26495) UID: 0 2024-09-22T01:16:51.596 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:51.591+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:51.625 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:51.625 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:51.639 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-22T01:16:51.640 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is set 2024-09-22T01:16:51.697 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:51.691+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:51.781 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:51.781 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:51.781 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:51.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-22T01:16:51.798 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-22T01:16:51.794+0000 7f2975556640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26497) UID: 0 2024-09-22T01:16:51.898 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-22T01:16:51.894+0000 7fc6130d0640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26491) UID: 0 2024-09-22T01:16:51.962 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:51.962 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:117358 3.1b 3:df60369f:::rbd_data.1124dbf5445a.0000000000000038:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1478034~34848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.015498132999994, 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005740968, 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117358 2024-09-22T01:16:51.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943353+0000", 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943355+0000", 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5069999999999999e-06 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943371+0000", 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6671999999999999e-05 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943386+0000", 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4902999999999999e-05 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943394+0000", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2029999999999992e-06 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943448+0000", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3638999999999999e-05 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:31.943456+0000", 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0409999999999998e-06 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.941748+0000", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.99829206800000003 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942290+0000", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054150900000000002 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.942738+0000", 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044784300000000001 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.945260+0000", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025227959999999999 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948819+0000", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035588009999999999 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.948979+0000", 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000159934 2024-09-22T01:16:51.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:32.949094+0000", 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115052 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123287 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.859648832000005, 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037284017000000003, 2024-09-22T01:16:51.969 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123287 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099202+0000", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099203+0000", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099205+0000", 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9999999999999999e-06 2024-09-22T01:16:51.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099211+0000", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4309999999999999e-06 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099217+0000", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3709999999999999e-06 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136300+0000", 2024-09-22T01:16:51.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037083637000000003 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136385+0000", 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5167000000000006e-05 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136486+0000", 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100994 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123288 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.859627502999999, 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037435782000000001, 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123288 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099224+0000", 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-22T01:16:51.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099227+0000", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.571e-06 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099233+0000", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2389999999999999e-06 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099238+0000", 2024-09-22T01:16:51.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1989999999999997e-06 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136539+0000", 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037300737 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136597+0000", 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8347000000000002e-05 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136660+0000", 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2242999999999993e-05 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123290 3.3 3:c4fb28b5:::rbd_data.1124ddce6057.0000000000000000:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.859164430000007, 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037277145999999997, 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123290 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099687+0000", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8800000000000003e-07 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099690+0000", 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099696+0000", 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8179999999999998e-06 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099702+0000", 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4349999999999996e-06 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136852+0000", 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037150576999999997 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136906+0000", 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3350999999999998e-05 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.136964+0000", 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8247e-05 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123419 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1097728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.859137559999994, 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037643358000000002, 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123419 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:51.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099714+0000", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099716+0000", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2740000000000002e-06 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099723+0000", 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.793e-06 2024-09-22T01:16:51.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099728+0000", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1590000000000001e-06 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137003+0000", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037274622 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137053+0000", 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9764e-05 2024-09-22T01:16:51.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137357+0000", 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030423499999999998 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123420 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1163264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.859114051999995, 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038023064000000002, 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123420 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099737+0000", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099738+0000", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099740+0000", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0109999999999999e-06 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099746+0000", 2024-09-22T01:16:51.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1249999999999998e-06 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099758+0000", 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1953000000000001e-05 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137401+0000", 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037642793000000001 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137483+0000", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2769000000000007e-05 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137760+0000", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000276865 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123421 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1228800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:51.991 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.858917106000007, 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038187296000000003, 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123421 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:51.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099934+0000", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099935+0000", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7899999999999998e-07 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099938+0000", 2024-09-22T01:16:51.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1140000000000002e-06 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099947+0000", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.397e-06 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099954+0000", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.421e-06 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137794+0000", 2024-09-22T01:16:51.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037840367999999999 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.137848+0000", 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000002e-05 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138121+0000", 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00027296699999999999 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:51.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123422 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1294336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.858889039999994, 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038620423000000001, 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123422 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:51.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099962+0000", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099963+0000", 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200000000000001e-07 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099965+0000", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9240000000000001e-06 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099971+0000", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4160000000000001e-06 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.099976+0000", 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138225+0000", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.038248351 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138337+0000", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011206599999999999 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138583+0000", 2024-09-22T01:16:51.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000246006 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123423 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857647600999996, 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039968750999999997, 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123423 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101204+0000", 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101205+0000", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3799999999999996e-07 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101209+0000", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101220+0000", 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0977e-05 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101228+0000", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4789999999999999e-06 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138625+0000", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.037396789 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.138684+0000", 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9623000000000002e-05 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141172+0000", 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024882340000000002 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123424 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1425408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857609835000005, 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040268160999999997, 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123424 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101241+0000", 2024-09-22T01:16:52.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101242+0000", 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.51e-07 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101244+0000", 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5239999999999999e-06 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101251+0000", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9739999999999996e-06 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101258+0000", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141208+0000", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.039950560000000003 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141265+0000", 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6910000000000002e-05 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141510+0000", 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00024443499999999999 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123425 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857585881000006, 2024-09-22T01:16:52.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040536038000000003, 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123425 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101265+0000", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101266+0000", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.03e-07 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101268+0000", 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101274+0000", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3690000000000003e-06 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101279+0000", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9350000000000002e-06 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141541+0000", 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040261763999999998 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141586+0000", 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5111999999999998e-05 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141801+0000", 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021551299999999999 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.012 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123426 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857564795000002, 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040709136999999999, 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123426 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:52.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101286+0000", 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101287+0000", 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6400000000000003e-07 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101289+0000", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.244e-06 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101296+0000", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5980000000000002e-06 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101301+0000", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141835+0000", 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040534489 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141908+0000", 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3201000000000006e-05 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.141996+0000", 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7311999999999993e-05 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123427 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857544331, 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040887231000000003, 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123427 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101307+0000", 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101309+0000", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.187e-06 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101316+0000", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2940000000000002e-06 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101321+0000", 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9289999999999997e-06 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142031+0000", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040710532000000001 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142111+0000", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9962000000000005e-05 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142194+0000", 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2976e-05 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.020 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123428 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857524310000002, 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094202000000003, 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123428 2024-09-22T01:16:52.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101327+0000", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9799999999999999e-07 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101329+0000", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101336+0000", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2450000000000003e-06 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101340+0000", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3279999999999999e-06 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142277+0000", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.040936815000000001 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142355+0000", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8213000000000005e-05 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142421+0000", 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6113999999999998e-05 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123429 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857505111999998, 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041253286, 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.025 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123429 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101346+0000", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101349+0000", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0779999999999998e-06 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101355+0000", 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5969999999999997e-06 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101360+0000", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9579999999999998e-06 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142455+0000", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041094952999999997 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142511+0000", 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6033999999999999e-05 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142599+0000", 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8327999999999999e-05 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123430 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857483999999999, 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041425269000000001, 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123430 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101367+0000", 2024-09-22T01:16:52.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101368+0000", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101370+0000", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1500000000000002e-06 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101376+0000", 2024-09-22T01:16:52.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101381+0000", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142630+0000", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041249566000000001 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142721+0000", 2024-09-22T01:16:52.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0625999999999996e-05 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142793+0000", 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1447999999999998e-05 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123431 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1884160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857463408000001, 2024-09-22T01:16:52.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041579423999999997, 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123431 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101388+0000", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7300000000000001e-07 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101390+0000", 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.153e-06 2024-09-22T01:16:52.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101397+0000", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101402+0000", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6709999999999998e-06 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142835+0000", 2024-09-22T01:16:52.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041433098000000002 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142896+0000", 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1579000000000001e-05 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142967+0000", 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0883999999999994e-05 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.037 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123432 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857442047000006, 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041687138999999998, 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123432 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101409+0000", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101410+0000", 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4799999999999999e-07 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101412+0000", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7199999999999998e-06 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101419+0000", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3430000000000001e-06 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101423+0000", 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:52.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.142999+0000", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041575426999999998 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143043+0000", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4597e-05 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143096+0000", 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2874999999999998e-05 2024-09-22T01:16:52.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123433 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2015232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857420732999998, 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041791864999999997, 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123433 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101431+0000", 2024-09-22T01:16:52.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9400000000000001e-07 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101433+0000", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2730000000000001e-06 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101440+0000", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4169999999999997e-06 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101444+0000", 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7720000000000002e-06 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143126+0000", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041681942999999999 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143169+0000", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2750999999999998e-05 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143222+0000", 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3315e-05 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:123434 3.3 3:c14c7490:::rbd_data.1124ddce6057.0000000000000009:2d [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.857401714999995, 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041882895000000003, 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 123434 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101450+0000", 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6600000000000003e-07 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101464+0000", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4018e-05 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101470+0000", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0410000000000003e-06 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.101475+0000", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0930000000000002e-06 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143241+0000", 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041765806000000003 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143284+0000", 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3322000000000003e-05 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:37.143332+0000", 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8349000000000001e-05 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.049 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-22T01:16:52.051 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:52.047+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:52.151 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:52.147+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:52.218 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:52.219 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-22T01:16:52.252 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:52.251+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:52.353 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:52.347+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:52.355 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:52.355 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:52.355 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.369 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-22T01:16:52.454 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-22T01:16:52.450+0000 7f1fbdaeb640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26493) UID: 0 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4207.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 181.50295210900001, 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19612374900000001, 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4207", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.150:0/1785509670", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984468+0000", 2024-09-22T01:16:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984471+0000", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984498+0000", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7106000000000001e-05 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984521+0000", 2024-09-22T01:16:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3292999999999999e-05 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984538+0000", 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6909999999999999e-05 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984689+0000", 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000151232 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:50.984707+0000", 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7586999999999999e-05 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177679+0000", 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.19297207699999999 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.177962+0000", 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028350100000000002 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.178361+0000", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039843000000000002 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180073+0000", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017118840000000001 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180407+0000", 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033396400000000002 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180509+0000", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102339 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:51.180592+0000", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2682999999999994e-05 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57903 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4525056~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.29720445, 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056028804000000001, 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57903 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190216+0000", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0900000000000002e-07 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190218+0000", 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-22T01:16:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190225+0000", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4119999999999998e-06 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190230+0000", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0939999999999998e-06 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246073+0000", 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055842633000000003 2024-09-22T01:16:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246124+0000", 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1492000000000001e-05 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246244+0000", 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012045299999999999 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57904 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4533248~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.297171678, 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05616699, 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57904 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190248+0000", 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190249+0000", 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190251+0000", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.266e-06 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190258+0000", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190263+0000", 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246275+0000", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05601138 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246315+0000", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0704000000000002e-05 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246415+0000", 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010008199999999999 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57905 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4541440~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.297150214, 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056351891000000001, 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57905 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190270+0000", 2024-09-22T01:16:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9100000000000004e-07 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190273+0000", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3879999999999998e-06 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190279+0000", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2129999999999996e-06 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190284+0000", 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-22T01:16:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246446+0000", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056162155999999998 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246501+0000", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4768000000000001e-05 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246622+0000", 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120748 2024-09-22T01:16:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57906 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4549632~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.297129271, 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056493100999999997, 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57906 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190291+0000", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190293+0000", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190300+0000", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3459999999999999e-06 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190304+0000", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246651+0000", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056347269999999998 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246730+0000", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8619000000000001e-05 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246784+0000", 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3915000000000001e-05 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57907 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4557824~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.297109363, 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056664063000000001, 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57907 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190311+0000", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190313+0000", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190320+0000", 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.353e-06 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190324+0000", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246842+0000", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056517801999999999 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246920+0000", 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7435999999999995e-05 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.246975+0000", 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5083000000000001e-05 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57908 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4566016~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.29708898200001, 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056823064999999999, 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57908 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190331+0000", 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190334+0000", 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190340+0000", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190346+0000", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7300000000000002e-06 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247004+0000", 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056657573000000003 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247100+0000", 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6559999999999997e-05 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247154+0000", 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4073999999999998e-05 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57909 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4574208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.29706704500001, 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.057049829000000003, 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57909 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190353+0000", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190751+0000", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00039808299999999998 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190758+0000", 2024-09-22T01:16:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2819999999999999e-06 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190843+0000", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4079000000000006e-05 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190850+0000", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000003e-06 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247233+0000", 2024-09-22T01:16:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056383023999999997 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247276+0000", 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999998e-05 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247403+0000", 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012691300000000001 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:57910 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 4582400~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e68)", 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.296557464, 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056724950000000003, 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 57910 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190863+0000", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0500000000000003e-07 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190866+0000", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0740000000000001e-06 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190874+0000", 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0630000000000006e-06 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.190881+0000", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3829999999999997e-06 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247430+0000", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056548963000000001 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247470+0000", 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0479999999999999e-05 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:53.247588+0000", 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000117382 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73137 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1144832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.565176417, 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055936421, 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73137 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922244+0000", 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922247+0000", 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922255+0000", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922261+0000", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978053+0000", 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055791896000000001 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978122+0000", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9605999999999997e-05 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978180+0000", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7785000000000003e-05 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73139 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1153024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.56514211299999, 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056060534000000002, 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73139 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922278+0000", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922279+0000", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.82e-07 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922281+0000", 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5189999999999999e-06 2024-09-22T01:16:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922288+0000", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9859999999999997e-06 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922293+0000", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3129999999999998e-06 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978208+0000", 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055914420999999999 2024-09-22T01:16:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978280+0000", 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2541999999999998e-05 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978339+0000", 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8171000000000003e-05 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73141 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.565071469, 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056142681, 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73141 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922349+0000", 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6199999999999997e-07 2024-09-22T01:16:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922353+0000", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5370000000000002e-06 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922361+0000", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2269999999999993e-06 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922367+0000", 2024-09-22T01:16:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0499999999999997e-06 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978366+0000", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055999050000000002 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978434+0000", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8269000000000007e-05 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978491+0000", 2024-09-22T01:16:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6885999999999997e-05 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73143 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1169408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.564919765, 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056153452999999999, 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73143 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922500+0000", 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922501+0000", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922505+0000", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1590000000000003e-06 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922515+0000", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9890000000000004e-06 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922522+0000", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7139999999999996e-06 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978519+0000", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055996508 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978589+0000", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0711999999999997e-05 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978654+0000", 2024-09-22T01:16:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4523000000000005e-05 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73145 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1177600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.564704228, 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.05609778, 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73145 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922716+0000", 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922717+0000", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.71e-07 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922721+0000", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0609999999999997e-06 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922731+0000", 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0242e-05 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922737+0000", 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4200000000000004e-06 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978684+0000", 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055946993 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978756+0000", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1574000000000003e-05 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978814+0000", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7618999999999998e-05 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73147 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1121280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.564491003, 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056079102999999998, 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73147 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922929+0000", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922930+0000", 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.023e-06 2024-09-22T01:16:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922935+0000", 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6870000000000002e-06 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922945+0000", 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0525e-05 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.922953+0000", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2760000000000003e-06 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978842+0000", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055889084999999998 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.978914+0000", 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2352000000000001e-05 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979008+0000", 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4154999999999996e-05 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73149 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1129472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.56429287900001, 2024-09-22T01:16:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056035929999999998, 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73149 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923127+0000", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923128+0000", 2024-09-22T01:16:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.09e-07 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923132+0000", 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1219999999999997e-06 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923143+0000", 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0740999999999999e-05 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923150+0000", 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.827e-06 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979037+0000", 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055886955000000002 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979105+0000", 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8430999999999996e-05 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979163+0000", 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8044999999999997e-05 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73151 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1137664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.564089087, 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056025434999999998, 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73151 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923331+0000", 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923332+0000", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923336+0000", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1459999999999998e-06 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923346+0000", 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0064000000000001e-05 2024-09-22T01:16:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923353+0000", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979191+0000", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055838204000000002 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979262+0000", 2024-09-22T01:16:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1344e-05 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979356+0000", 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3978e-05 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73153 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.563838262, 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055930530999999999, 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73153 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923582+0000", 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923583+0000", 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2299999999999999e-07 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923587+0000", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2350000000000001e-06 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923597+0000", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0421000000000001e-05 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923604+0000", 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7630000000000003e-06 2024-09-22T01:16:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979385+0000", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055780429999999999 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979454+0000", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9867e-05 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979512+0000", 2024-09-22T01:16:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7892000000000001e-05 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73155 3.1a 3:5955da27:::rbd_data.112424dbdb5c.0000000000000004:18 [read 1154048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.563619242, 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055903296999999998, 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73155 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923801+0000", 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923802+0000", 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9800000000000003e-07 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923806+0000", 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2379999999999999e-06 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923816+0000", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7359999999999996e-06 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.923822+0000", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8159999999999996e-06 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979540+0000", 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.055718034 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979613+0000", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2132000000000007e-05 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.979704+0000", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1543000000000001e-05 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:150139 3.6 3:622980ff:::rbd_data.1124b08b8687.0000000000000151:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 61.876581633999997, 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92609944399999999, 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150139 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610838+0000", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610840+0000", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6819999999999999e-06 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610851+0000", 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1291000000000001e-05 2024-09-22T01:16:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610872+0000", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0052999999999999e-05 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610880+0000", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5269999999999998e-06 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610953+0000", 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2934000000000003e-05 2024-09-22T01:16:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:50.610963+0000", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0458999999999999e-05 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531402+0000", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92043835799999996 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531615+0000", 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021305799999999999 2024-09-22T01:16:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.531903+0000", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00028862499999999998 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.533833+0000", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019296000000000001 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536795+0000", 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029622049999999999 2024-09-22T01:16:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536876+0000", 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1014999999999996e-05 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:51.536938+0000", 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1637000000000006e-05 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:52.583 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.585 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-22T01:16:52.585 INFO:tasks.ceph.osd.5.smithi150.stderr:2024-09-22T01:16:52.579+0000 7f4206891640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26569) UID: 0 2024-09-22T01:16:52.650 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 6, 7, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-22T01:16:52.651 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-22T01:16:52.651 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-22T01:16:52.651 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-22T01:16:52.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-09-22T01:16:52.686 INFO:tasks.ceph.osd.7.smithi150.stderr:2024-09-22T01:16:52.679+0000 7f5ba687f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26573) UID: 0 2024-09-22T01:16:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:52.781 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:52.781 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.786 INFO:tasks.ceph.osd.6.smithi150.stderr:2024-09-22T01:16:52.783+0000 7f6faccc9640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26570) UID: 0 2024-09-22T01:16:52.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-22T01:16:52.806 INFO:teuthology.orchestra.run.smithi096.stdout:All operations completed A-OK! 2024-09-22T01:16:52.856 DEBUG:teuthology.parallel:result is None 2024-09-22T01:16:52.856 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-22T01:16:52.870 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-22T01:16:52.894 INFO:tasks.thrashosds:joining thrashosds 2024-09-22T01:16:52.917 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:52.917 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:52.917 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:52.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:52.917 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:52.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72760 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.14992922899999, 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090047885999999994, 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72760 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896060+0000", 2024-09-22T01:16:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896062+0000", 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0219999999999999e-06 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896068+0000", 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896073+0000", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983576+0000", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087503191999999994 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.983649+0000", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2716e-05 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986107+0000", 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024583579999999999 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72774 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 69632~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.149764873, 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090126955999999994, 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72774 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896224+0000", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896895+0000", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067086299999999995 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896900+0000", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1510000000000003e-06 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896911+0000", 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1005000000000001e-05 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.896918+0000", 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5190000000000003e-06 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.985931+0000", 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089012497999999995 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986037+0000", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106011 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.986351+0000", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031390900000000002 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72798 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 103424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.14507309699999, 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089926830999999999, 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72798 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900916+0000", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6100000000000001e-07 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900918+0000", 2024-09-22T01:16:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3159999999999999e-06 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900925+0000", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5239999999999997e-06 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900930+0000", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9099999999999996e-06 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990592+0000", 2024-09-22T01:16:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089662139000000002 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990675+0000", 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2806000000000006e-05 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990843+0000", 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000167675 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72800 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 111616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.14503554300001, 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090157138999999997, 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72800 2024-09-22T01:16:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900953+0000", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900954+0000", 2024-09-22T01:16:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5499999999999998e-07 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900961+0000", 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4880000000000001e-06 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900970+0000", 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5699999999999993e-06 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900976+0000", 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2269999999999998e-06 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990875+0000", 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089898618999999999 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.990958+0000", 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3374999999999994e-05 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991110+0000", 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015220500000000001 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72802 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 119808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.145003381, 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090417029999999995, 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72802 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900985+0000", 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900986+0000", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900989+0000", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7580000000000001e-06 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.900996+0000", 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7939999999999997e-06 2024-09-22T01:16:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901000+0000", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9509999999999997e-06 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991141+0000", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090140440000000002 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991228+0000", 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7556e-05 2024-09-22T01:16:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991402+0000", 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000173936 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72804 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 128000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.144981044, 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090657714, 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72804 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901008+0000", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7899999999999999e-07 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901011+0000", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.401e-06 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901017+0000", 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2999999999999998e-06 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901022+0000", 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991433+0000", 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090411367000000006 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991515+0000", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1390000000000005e-05 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991665+0000", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000150815 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72806 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 71680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.144959684, 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090937567999999996, 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72806 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901029+0000", 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901030+0000", 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901032+0000", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4420000000000001e-06 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901039+0000", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7299999999999999e-06 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901044+0000", 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991700+0000", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090656478999999998 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991786+0000", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5805999999999998e-05 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991967+0000", 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018058500000000001 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72807 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.144938494, 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091181708, 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72807 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.901050+0000", 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904734+0000", 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0036836719999999998 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904738+0000", 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1470000000000003e-06 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904747+0000", 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2699999999999993e-06 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904755+0000", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7819999999999994e-06 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.991997+0000", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.087241949999999999 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992078+0000", 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1332e-05 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.992232+0000", 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015355499999999999 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72822 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.141033001, 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090088391000000004, 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72822 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904956+0000", 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.75e-07 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904959+0000", 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.305e-06 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904965+0000", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7190000000000004e-06 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904971+0000", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5210000000000002e-06 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994829+0000", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089857973999999993 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.994905+0000", 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5896999999999995e-05 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995044+0000", 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0001395 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72824 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.14101082000001, 2024-09-22T01:16:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090334751000000005, 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72824 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904978+0000", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0900000000000002e-07 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904981+0000", 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0700000000000001e-06 2024-09-22T01:16:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904987+0000", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7329999999999997e-06 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904992+0000", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.921e-06 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995073+0000", 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080568999999999 2024-09-22T01:16:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995182+0000", 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000109069 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995313+0000", 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013087999999999999 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72826 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.140989542, 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090575615999999998, 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72826 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.904999+0000", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905000+0000", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4999999999999998e-07 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905002+0000", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905009+0000", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5189999999999998e-06 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905014+0000", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3550000000000004e-06 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995340+0000", 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090325638999999999 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995424+0000", 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4919000000000005e-05 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995575+0000", 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015042699999999999 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72828 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.140967195, 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090844937000000001, 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72828 2024-09-22T01:16:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905022+0000", 2024-09-22T01:16:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5400000000000002e-07 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905024+0000", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905031+0000", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6499999999999999e-06 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905036+0000", 2024-09-22T01:16:53.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1930000000000001e-06 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995630+0000", 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090593730999999997 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995716+0000", 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6598e-05 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995867+0000", 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015014999999999999 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72831 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.140945519, 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091081550999999997, 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72831 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905043+0000", 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905044+0000", 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905046+0000", 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2419999999999999e-06 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905052+0000", 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4690000000000002e-06 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905058+0000", 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0869999999999997e-06 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995896+0000", 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090838110999999999 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.995978+0000", 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2309000000000001e-05 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996125+0000", 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000146885 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:72833 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.140924097, 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091320935000000006, 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 72833 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905065+0000", 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5699999999999998e-07 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905067+0000", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1660000000000001e-06 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905074+0000", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6590000000000001e-06 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.905079+0000", 2024-09-22T01:16:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2519999999999999e-06 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996151+0000", 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091071556999999997 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996249+0000", 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7971000000000001e-05 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.996386+0000", 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000136873 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73062 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.13428535200001, 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090080099999999996, 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73062 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911703+0000", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911704+0000", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0400000000000005e-07 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911708+0000", 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1289999999999998e-06 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911719+0000", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0254e-05 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911726+0000", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.17e-06 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001521+0000", 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089795106 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001643+0000", 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012215199999999999 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001784+0000", 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014048500000000001 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73064 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.134068605, 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090141373999999996, 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73064 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911920+0000", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911921+0000", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3399999999999998e-07 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911925+0000", 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7479999999999999e-06 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911936+0000", 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1126e-05 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.911944+0000", 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0360000000000006e-06 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001815+0000", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089871432000000001 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.001902+0000", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6427000000000004e-05 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002062+0000", 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015977100000000001 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73066 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 985088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.133685978, 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090134222, 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73066 2024-09-22T01:16:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912303+0000", 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912304+0000", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8400000000000003e-07 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912308+0000", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912318+0000", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0492000000000001e-05 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912325+0000", 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8630000000000002e-06 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002096+0000", 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089770655000000005 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002177+0000", 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1143999999999994e-05 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002437+0000", 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00026018900000000003 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73068 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 993280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.133280438, 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139408000000004, 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73068 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912708+0000", 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912709+0000", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000003e-07 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912714+0000", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912906+0000", 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019195700000000001 2024-09-22T01:16:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.912945+0000", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8622999999999999e-05 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002472+0000", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089527714999999994 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002611+0000", 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013846400000000001 2024-09-22T01:16:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.002848+0000", 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000237023 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73092 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 1027072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.13006397700001, 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.090139101999999999, 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73092 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915925+0000", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0199999999999998e-07 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915926+0000", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3459999999999999e-06 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915930+0000", 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.915932+0000", 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7429999999999998e-06 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005802+0000", 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.089869619999999997 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.005912+0000", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110325 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.006064+0000", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015148599999999999 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:73508 3.16 3:6ad87512:::rbd_data.112424dbdb5c.0000000000000002:18 [read 448512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 118.093278582, 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.091172486999999997, 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 73508 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952710+0000", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952711+0000", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952724+0000", 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3284e-05 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952732+0000", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:54.952738+0000", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7089999999999999e-06 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036021+0000", 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.083283733999999998 2024-09-22T01:16:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.036170+0000", 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014898 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:55.043883+0000", 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0077124120000000001 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.135 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:53.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-22T01:16:53.264 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:53.264 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-22T01:16:53.264 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:53.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-22T01:16:53.399 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:53.399 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-22T01:16:53.399 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-22T01:16:53.399 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-22T01:16:53.399 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:53.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4220.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 178.50966636000001, 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32413626000000001, 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4220", 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/122750337", 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019854+0000", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019857+0000", 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019863+0000", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6560000000000003e-06 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019880+0000", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6413999999999999e-05 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019892+0000", 2024-09-22T01:16:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2639999999999999e-05 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019982+0000", 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9361e-05 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.019991+0000", 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6819999999999998e-06 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340440+0000", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.32044830499999999 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.340953+0000", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00051342 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.341304+0000", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000351023 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-22T01:16:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.342804+0000", 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001499998 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343806+0000", 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001002303 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343904+0000", 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7583000000000004e-05 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:13:55.343990+0000", 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6274999999999996e-05 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:23204 3.14 3:2f655298:::rbd_header.11249227f48c:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 138.96210256399999, 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95179165300000002, 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 23204 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567418+0000", 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567420+0000", 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567430+0000", 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0352e-05 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567453+0000", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2568000000000001e-05 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567464+0000", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0978999999999999e-05 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567544+0000", 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9909999999999999e-05 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:34.567555+0000", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1620999999999999e-05 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518184+0000", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.95062884299999995 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.518524+0000", 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00033980300000000001 2024-09-22T01:16:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:14:35.519209+0000", 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068544299999999999 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160166 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 466944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.266467327999997, 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115178461, 2024-09-22T01:16:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160166 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263053+0000", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263055+0000", 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6980000000000001e-06 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263060+0000", 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263064+0000", 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0069999999999999e-06 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.377950+0000", 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114885865 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378082+0000", 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013210199999999999 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378231+0000", 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014958500000000001 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160167 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 989184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.266451752000002, 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115674285, 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160167 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263069+0000", 2024-09-22T01:16:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263070+0000", 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4479999999999999e-06 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263075+0000", 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6129999999999997e-06 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263080+0000", 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6399999999999996e-06 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378321+0000", 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11524171 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378512+0000", 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019034499999999999 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378743+0000", 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023119199999999999 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160168 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 483328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.266435924, 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116038742, 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160168 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263084+0000", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263085+0000", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.34e-07 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263086+0000", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.514e-06 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263091+0000", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6260000000000003e-06 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263095+0000", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8940000000000003e-06 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378847+0000", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115752567 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.378976+0000", 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012830900000000001 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379123+0000", 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147498 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160169 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1005568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.266420930999999, 2024-09-22T01:16:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11653605, 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160169 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263099+0000", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263100+0000", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7199999999999999e-07 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263101+0000", 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5149999999999999e-06 2024-09-22T01:16:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263106+0000", 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6330000000000004e-06 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263110+0000", 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8859999999999997e-06 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379216+0000", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116106207 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379405+0000", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018844200000000001 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379635+0000", 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000230895 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160170 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.266405741, 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116900905, 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160170 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.263115+0000", 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264937+0000", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018220070000000001 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264944+0000", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6739999999999997e-06 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264955+0000", 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0387999999999999e-05 2024-09-22T01:16:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.264961+0000", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2430000000000002e-06 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379742+0000", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11478102900000001 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.379866+0000", 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012440199999999999 2024-09-22T01:16:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380016+0000", 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014916199999999999 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160171 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1021952~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.264514151, 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115522942, 2024-09-22T01:16:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160171 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265006+0000", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265007+0000", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265009+0000", 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1150000000000001e-06 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265015+0000", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.241e-06 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265020+0000", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.386e-06 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380106+0000", 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115086678 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380297+0000", 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000190302 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380529+0000", 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023262099999999999 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160172 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.264494300000003, 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11590674500000001, 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160172 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265026+0000", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265027+0000", 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265028+0000", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6470000000000001e-06 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265033+0000", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919999999999998e-06 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265038+0000", 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.977e-06 2024-09-22T01:16:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380653+0000", 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11561484800000001 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380781+0000", 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012814200000000001 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.380933+0000", 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015169099999999999 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160173 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.264476858000002, 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11637212399999999, 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160173 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265043+0000", 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265044+0000", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265045+0000", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5230000000000001e-06 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265050+0000", 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7210000000000002e-06 2024-09-22T01:16:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265054+0000", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8750000000000002e-06 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381021+0000", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115967025 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381196+0000", 2024-09-22T01:16:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505900000000001 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381416+0000", 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00021956999999999999 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160174 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 532480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.264461402999999, 2024-09-22T01:16:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116705745, 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160174 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265059+0000", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265065+0000", 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4090000000000002e-06 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265070+0000", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0839999999999999e-06 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265074+0000", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0119999999999999e-06 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381492+0000", 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11641849999999999 2024-09-22T01:16:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381635+0000", 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142275 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381765+0000", 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013010300000000001 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160175 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1054720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.264440368999999, 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.117196034, 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160175 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.265080+0000", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268592+0000", 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003512472 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268597+0000", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6600000000000003e-06 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268606+0000", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2350000000000005e-06 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268612+0000", 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7169999999999996e-06 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.381855+0000", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.113242756 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382047+0000", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000191981 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.382276+0000", 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000229213 2024-09-22T01:16:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160179 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.260778422000001, 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115362886, 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160179 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268742+0000", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268744+0000", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.595e-06 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268750+0000", 2024-09-22T01:16:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7590000000000003e-06 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268754+0000", 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9729999999999999e-06 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383684+0000", 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.114930114 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.383873+0000", 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018971600000000001 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384105+0000", 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023137799999999999 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160180 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 581632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.260761817000002, 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11584968599999999, 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160180 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268759+0000", 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4299999999999999e-07 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268760+0000", 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5200000000000001e-06 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268765+0000", 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268769+0000", 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.98e-06 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384182+0000", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115412289 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384369+0000", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018727899999999999 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384608+0000", 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023941699999999999 2024-09-22T01:16:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160181 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1103872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.260746456, 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116242703, 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160181 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268774+0000", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.41e-07 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268776+0000", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4899999999999999e-06 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268780+0000", 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268784+0000", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8670000000000004e-06 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384695+0000", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11591104300000001 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.384838+0000", 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000142788 2024-09-22T01:16:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385017+0000", 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017848000000000001 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160182 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 598016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.260731292000003, 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116740161, 2024-09-22T01:16:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160182 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.268789+0000", 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273003+0000", 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0042141339999999996 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273009+0000", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273022+0000", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2913e-05 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273029+0000", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5089999999999996e-06 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385109+0000", 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.112079663 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385303+0000", 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019432499999999999 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.385529+0000", 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022592099999999999 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160188 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 647168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.256369358999997, 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11536550600000001, 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160188 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273151+0000", 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2500000000000001e-07 2024-09-22T01:16:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273153+0000", 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5179999999999999e-06 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273158+0000", 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6609999999999999e-06 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273162+0000", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-06 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388098+0000", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11493666499999999 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388291+0000", 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019279900000000001 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388517+0000", 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022547800000000001 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160189 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1169408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.256353865000001, 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115787379, 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160189 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273166+0000", 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273167+0000", 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273168+0000", 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5799999999999999e-06 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273173+0000", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273177+0000", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9199999999999997e-06 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388625+0000", 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11544792600000001 2024-09-22T01:16:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388782+0000", 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015664800000000001 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.388954+0000", 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017227400000000001 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160190 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 663552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.256338323999998, 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.116265724, 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160190 2024-09-22T01:16:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.273182+0000", 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.276998+0000", 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003816074 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277008+0000", 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0267e-05 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277033+0000", 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5041999999999998e-05 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277048+0000", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.433e-05 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389052+0000", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.11200427 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389230+0000", 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017767999999999999 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.389448+0000", 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000218061 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4388.0:160197 3.14 3:2e1e9ac4:::rbd_data.11246ae4e8f8.0000000000000001:32 [read 1234944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.252249755999998, 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.115558328, 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-22T01:16:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4388", 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 160197 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.277271+0000", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280780+0000", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035096649999999999 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280790+0000", 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8030000000000008e-06 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280817+0000", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7143000000000001e-05 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.280832+0000", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4839e-05 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392490+0000", 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.111657823 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392654+0000", 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000163735 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-22T01:15:55.392829+0000", 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017532 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-22T01:16:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-22T01:16:53.623 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:16:53.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-22T01:16:53.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-22T01:16:53.676 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-22T01:16:53.758 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:53.759 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:53.759 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:53.759 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:53.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-22T01:16:53.895 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:53.895 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:53.895 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:53.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-22T01:16:54.035 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.035 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:54.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:54.035 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:54.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:97790 3.2 3:4784744f:::rbd_data.1124ccc9ab83.0000000000000055:head [stat,delete] snapc 1e=[1e] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 109.22553295900001, 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074886848000000006, 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 97790 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805830+0000", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805832+0000", 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:54.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805846+0000", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4319e-05 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805894+0000", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8194e-05 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.805918+0000", 2024-09-22T01:16:54.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3493e-05 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806074+0000", 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000156377 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.806439+0000", 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036430099999999998 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:04.880717+0000", 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.074277857000000003 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:104478 3.1 3:85ba06ba:::rbd_data.1124ee2fa883.0000000000000010:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 101.01814362, 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77335114699999996, 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 104478 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013219+0000", 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013221+0000", 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.86e-06 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013229+0000", 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013254+0000", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4706e-05 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.013272+0000", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889000000000001e-05 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014125+0000", 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00085348800000000001 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "waiting for peered", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.014150+0000", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4612000000000001e-05 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786311+0000", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.77216111399999998 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:13.786570+0000", 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000258965 2024-09-22T01:16:54.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194356 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345534802000003, 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063214355, 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.045 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194356 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:54.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685828+0000", 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7099999999999997e-07 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685830+0000", 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0470000000000001e-06 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685837+0000", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2990000000000002e-06 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685841+0000", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9169999999999997e-06 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748878+0000", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063036393999999996 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.748957+0000", 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9003000000000002e-05 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749042+0000", 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5323999999999998e-05 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194359 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345514672, 2024-09-22T01:16:54.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063397326000000004, 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194359 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:54.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685848+0000", 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0399999999999996e-07 2024-09-22T01:16:54.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685851+0000", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1339999999999998e-06 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685857+0000", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.657e-06 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685862+0000", 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8289999999999998e-06 2024-09-22T01:16:54.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749072+0000", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063210410999999994 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749155+0000", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2429000000000006e-05 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749245+0000", 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0362e-05 2024-09-22T01:16:54.053 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194360 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345493842000003, 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063571744999999999, 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.054 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194360 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685869+0000", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.6100000000000002e-07 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685871+0000", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9120000000000001e-06 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685878+0000", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5300000000000002e-06 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685882+0000", 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7469999999999996e-06 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749276+0000", 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063393719000000001 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749355+0000", 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9294000000000003e-05 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749441+0000", 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5181999999999999e-05 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194361 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345473816999998, 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063749019000000004, 2024-09-22T01:16:54.058 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194361 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685889+0000", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0499999999999999e-07 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685891+0000", 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0829999999999998e-06 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685898+0000", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1990000000000003e-06 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685902+0000", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.95e-06 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749470+0000", 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063567345999999997 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749550+0000", 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0117000000000006e-05 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749638+0000", 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.7918999999999996e-05 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.062 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194362 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345453137, 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063934405, 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194362 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.063 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685909+0000", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685910+0000", 2024-09-22T01:16:54.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685912+0000", 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1710000000000001e-06 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685918+0000", 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1569999999999997e-06 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685924+0000", 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3499999999999996e-06 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749668+0000", 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063744540000000002 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749752+0000", 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3918999999999994e-05 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749844+0000", 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1926e-05 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194363 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 61440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.345430991000001, 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064067488000000006, 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.067 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194363 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.685932+0000", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686423+0000", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00049124399999999995 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686429+0000", 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0170000000000002e-06 2024-09-22T01:16:54.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686440+0000", 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1362000000000001e-05 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686446+0000", 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4319999999999998e-06 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.070 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749875+0000", 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063429725000000006 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749935+0000", 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9806000000000003e-05 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.749999+0000", 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3901999999999999e-05 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194364 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 81920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.344894379000003, 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063856552999999996, 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194364 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.072 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686468+0000", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686469+0000", 2024-09-22T01:16:54.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5600000000000001e-07 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686471+0000", 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3769999999999999e-06 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686479+0000", 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5290000000000003e-06 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686484+0000", 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5860000000000004e-06 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750070+0000", 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063586167999999998 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750206+0000", 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000135573 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750325+0000", 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00011886400000000001 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194365 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 100352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.344869729000003, 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064126737000000003, 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.076 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194365 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686493+0000", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5400000000000001e-07 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686496+0000", 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.683e-06 2024-09-22T01:16:54.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686503+0000", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0979999999999996e-06 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686521+0000", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7734e-05 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750360+0000", 2024-09-22T01:16:54.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063838791000000006 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750519+0000", 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00015956799999999999 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750620+0000", 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000100309 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194366 3.19 3:9d08054c:::rbd_data.1124b08b8687.0000000000000095:35 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.344833364000003, 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.064367674999999999, 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.081 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194366 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.686529+0000", 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689303+0000", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0027739290000000001 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689332+0000", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8602000000000002e-05 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689345+0000", 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.274e-05 2024-09-22T01:16:54.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689352+0000", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5639999999999999e-06 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750676+0000", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061323857000000002 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750775+0000", 2024-09-22T01:16:54.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.8881000000000004e-05 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.750897+0000", 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00012210199999999999 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194373 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 153600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.085 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.341883996, 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063195214, 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194373 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689479+0000", 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2899999999999999e-07 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689481+0000", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.708e-06 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.689486+0000", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4060000000000004e-06 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.690778+0000", 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.001291864 2024-09-22T01:16:54.088 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752516+0000", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.061737664999999997 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752591+0000", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5428999999999999e-05 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.752674+0000", 2024-09-22T01:16:54.089 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2812999999999995e-05 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194416 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 32768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340276031000002, 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063218363, 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.090 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194416 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.091 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691087+0000", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4700000000000002e-07 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691089+0000", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3070000000000001e-06 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691096+0000", 2024-09-22T01:16:54.092 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5540000000000003e-06 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691101+0000", 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0259999999999998e-06 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754153+0000", 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063052434000000004 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.093 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754228+0000", 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4423000000000002e-05 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754305+0000", 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7272000000000001e-05 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194417 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.094 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340255018999997, 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063381043999999997, 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194417 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.095 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691108+0000", 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7500000000000001e-07 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.096 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691110+0000", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0020000000000001e-06 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691117+0000", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7730000000000002e-06 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691129+0000", 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2099000000000001e-05 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.097 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754333+0000", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063204027999999995 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754402+0000", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9292e-05 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754489+0000", 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6475000000000001e-05 2024-09-22T01:16:54.098 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194418 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 86016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340225996999997, 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063533427000000003, 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.099 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194418 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.100 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691137+0000", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.06e-07 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691139+0000", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.322e-06 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691149+0000", 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0499999999999997e-06 2024-09-22T01:16:54.101 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691160+0000", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1926999999999999e-05 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754518+0000", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357152 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754591+0000", 2024-09-22T01:16:54.102 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3158999999999995e-05 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754670+0000", 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9311e-05 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194419 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 124928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340193649, 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063680215999999998, 2024-09-22T01:16:54.103 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194419 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.104 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691169+0000", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691170+0000", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5199999999999997e-07 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691172+0000", 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8090000000000001e-06 2024-09-22T01:16:54.105 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691204+0000", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2055999999999999e-05 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691219+0000", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4181e-05 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754698+0000", 2024-09-22T01:16:54.106 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063479422999999993 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754772+0000", 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3492999999999996e-05 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754849+0000", 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7701999999999996e-05 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.107 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194420 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 139264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340127529, 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063790847999999997, 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194420 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.108 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691235+0000", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691236+0000", 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6099999999999999e-07 2024-09-22T01:16:54.109 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691241+0000", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6940000000000003e-06 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691251+0000", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0479000000000001e-05 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691257+0000", 2024-09-22T01:16:54.110 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8749999999999997e-06 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754876+0000", 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063619027999999994 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.754948+0000", 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1532000000000006e-05 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.111 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755026+0000", 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8379000000000003e-05 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194421 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 178176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.340097120999999, 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063937157999999994, 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.112 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194421 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.691266+0000", 2024-09-22T01:16:54.113 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692610+0000", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0013447750000000001 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692614+0000", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.704e-06 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692625+0000", 2024-09-22T01:16:54.114 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.061e-05 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692630+0000", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2159999999999997e-06 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755054+0000", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062424526000000001 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755126+0000", 2024-09-22T01:16:54.115 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.1655000000000004e-05 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755203+0000", 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6672e-05 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194427 3.19 3:9b593d37:::rbd_data.1124b08b8687.0000000000000099:35 [read 299008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.338600497000002, 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063357034000000007, 2024-09-22T01:16:54.116 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194427 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.117 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692762+0000", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692763+0000", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1699999999999999e-07 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692765+0000", 2024-09-22T01:16:54.118 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2520000000000002e-06 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692772+0000", 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0539999999999997e-06 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692784+0000", 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2354999999999999e-05 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.119 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.755954+0000", 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063169863000000007 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756030+0000", 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6086000000000003e-05 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756119+0000", 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.9006999999999997e-05 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.120 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:194559 3.19 3.32a010b9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 54.338570136000001, 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.063367298000000002, 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 194559 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.121 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.692792+0000", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693157+0000", 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00036405999999999998 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.122 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693162+0000", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5430000000000001e-06 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693173+0000", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1277000000000001e-05 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.693181+0000", 2024-09-22T01:16:54.123 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5379999999999997e-06 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756149+0000", 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.062968515000000003 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:59.756160+0000", 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0365e-05 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.124 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.125 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.125 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.126 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-22T01:16:54.252 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.252 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:54.252 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:54.252 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.266 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-22T01:16:54.386 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:54.386 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:54.386 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-22T01:16:54.524 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.524 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:54.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:54.524 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:54.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103426 3.17 3:ea180d07:::rbd_data.1124772a770f.0000000000000000:1f [read 356352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.377287776, 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080935692000000004, 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103426 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.525 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144973+0000", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144974+0000", 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.097e-06 2024-09-22T01:16:54.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.144980+0000", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8629999999999996e-06 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145002+0000", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2178999999999999e-05 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145019+0000", 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.6668999999999999e-05 2024-09-22T01:16:54.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225476+0000", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080457382999999993 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225698+0000", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.00022139800000000001 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225909+0000", 2024-09-22T01:16:54.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000211103 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103427 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.377187836, 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080871856000000006, 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.529 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103427 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145073+0000", 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145074+0000", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.26e-06 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145086+0000", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1878999999999999e-05 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145108+0000", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1498999999999999e-05 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145122+0000", 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4267000000000001e-05 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225934+0000", 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080811923999999993 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.225945+0000", 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1027e-05 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103428 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37711894500001, 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080888885999999993, 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103428 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145142+0000", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145143+0000", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.206e-06 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145149+0000", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6500000000000001e-06 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145171+0000", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2028999999999999e-05 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145183+0000", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.272e-05 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226003+0000", 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080819572000000006 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226031+0000", 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7708999999999998e-05 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103433 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.377042882, 2024-09-22T01:16:54.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080840445999999996, 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103433 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145218+0000", 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145219+0000", 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.13e-06 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145226+0000", 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7569999999999998e-06 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145248+0000", 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1878999999999998e-05 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145262+0000", 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.434e-05 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226050+0000", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787827000000006 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226058+0000", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.5129999999999996e-06 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103434 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:54.540 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376980094, 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080847705000000006, 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103434 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145281+0000", 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145282+0000", 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1200000000000001e-06 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145287+0000", 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7629999999999997e-06 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145321+0000", 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3559000000000002e-05 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145347+0000", 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6291999999999998e-05 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226102+0000", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080755159000000007 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226128+0000", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5812e-05 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103452 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:54.544 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37634916, 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817981999999997, 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103452 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145912+0000", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145915+0000", 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2140000000000001e-06 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145924+0000", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1060000000000006e-06 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145932+0000", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.0859999999999996e-06 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226704+0000", 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080772153999999999 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226730+0000", 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5816000000000002e-05 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103457 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376320021, 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080815546000000002, 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.548 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103457 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145941+0000", 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6300000000000005e-07 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145944+0000", 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0900000000000001e-06 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145954+0000", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1360000000000003e-06 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145961+0000", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2230000000000001e-06 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226748+0000", 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080787033999999994 2024-09-22T01:16:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226756+0000", 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3999999999999992e-06 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103458 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376290696, 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080851234999999994, 2024-09-22T01:16:54.552 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103458 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145970+0000", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145971+0000", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0699999999999997e-07 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145974+0000", 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0860000000000002e-06 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145983+0000", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.0399999999999998e-06 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.145990+0000", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8469999999999998e-06 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226795+0000", 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080805718999999998 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226821+0000", 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.5936e-05 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103459 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37626006000001, 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080849596999999995, 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.556 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103459 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146001+0000", 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1900000000000002e-07 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146004+0000", 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.954e-06 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146020+0000", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5359999999999999e-05 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146027+0000", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3420000000000002e-06 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226840+0000", 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080812602999999997 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226850+0000", 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0719e-05 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103460 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376224392, 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080892537, 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.560 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103460 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146036+0000", 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146037+0000", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146040+0000", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2270000000000002e-06 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146049+0000", 2024-09-22T01:16:54.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1800000000000002e-06 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146056+0000", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7259999999999996e-06 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226901+0000", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845031999999997 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226929+0000", 2024-09-22T01:16:54.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7761999999999999e-05 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103465 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376195409, 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080890529000000003, 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.564 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103465 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146065+0000", 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146066+0000", 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.06e-07 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146069+0000", 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1640000000000001e-06 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146078+0000", 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.2690000000000005e-06 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146087+0000", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1200000000000008e-06 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226947+0000", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859918000000003 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.226956+0000", 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4519999999999997e-06 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.567 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103466 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.376161197, 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080937450999999994, 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.568 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103466 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146100+0000", 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146101+0000", 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.44e-06 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146106+0000", 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3290000000000001e-06 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146120+0000", 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4056999999999999e-05 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146133+0000", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3169e-05 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227007+0000", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080873811000000004 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227037+0000", 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.9645000000000002e-05 2024-09-22T01:16:54.571 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103467 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37609237700001, 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080895072999999998, 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.572 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103467 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146168+0000", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146169+0000", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.0869999999999999e-06 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146188+0000", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8522999999999999e-05 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146222+0000", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3717999999999998e-05 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146241+0000", 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9256999999999999e-05 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227055+0000", 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080813869999999996 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227063+0000", 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6179999999999995e-06 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.575 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103468 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.375991863, 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080859358000000006, 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103468 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.576 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146269+0000", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146270+0000", 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1829999999999999e-06 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.577 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146277+0000", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5089999999999999e-06 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146295+0000", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8393000000000002e-05 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146309+0000", 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3873000000000001e-05 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.578 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227106+0000", 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080797587000000004 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227128+0000", 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1812999999999999e-05 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103473 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.579 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37592370900001, 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818376999999997, 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103473 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:54.580 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146337+0000", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146339+0000", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.77e-06 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146346+0000", 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9999999999999999e-06 2024-09-22T01:16:54.581 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146370+0000", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4002000000000002e-05 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146399+0000", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.8858999999999999e-05 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227147+0000", 2024-09-22T01:16:54.582 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080748297999999996 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227155+0000", 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.4479999999999994e-06 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103474 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37583997900001, 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080806520000000007, 2024-09-22T01:16:54.583 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103474 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.584 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146421+0000", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146422+0000", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.327e-06 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146428+0000", 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0430000000000004e-06 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.585 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146450+0000", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1569e-05 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146463+0000", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3589e-05 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227204+0000", 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080740468999999995 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.586 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227227+0000", 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3523000000000002e-05 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103476 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37570610100001, 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080845309000000004, 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.587 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103476 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.588 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146555+0000", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146556+0000", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.1999999999999998e-07 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146561+0000", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7570000000000001e-06 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.589 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146586+0000", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4892000000000002e-05 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146601+0000", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4973e-05 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227359+0000", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080757654999999998 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.590 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227400+0000", 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.1112000000000002e-05 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103482 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37557925199999, 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080817795999999997, 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.591 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103482 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146681+0000", 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.592 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146682+0000", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5700000000000002e-07 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146687+0000", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4460000000000003e-06 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146700+0000", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.3667e-05 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.593 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146708+0000", 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6359999999999994e-06 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227476+0000", 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080768291000000006 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227499+0000", 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2999000000000001e-05 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.594 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103483 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.375542297, 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080807711000000004, 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103483 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.595 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146718+0000", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146719+0000", 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0999999999999998e-07 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.596 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146722+0000", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.2799999999999999e-06 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146741+0000", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8525999999999998e-05 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146752+0000", 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1513e-05 2024-09-22T01:16:54.597 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227518+0000", 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080765420000000004 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227526+0000", 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3620000000000006e-06 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.598 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:103484 3.17 3.e0b01857 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 105.37549065499999, 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080818996000000004, 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 103484 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.599 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146770+0000", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146771+0000", 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1400000000000001e-06 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.600 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146777+0000", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9630000000000003e-06 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146795+0000", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7915999999999999e-05 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.146808+0000", 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.326e-05 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.601 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227563+0000", 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.080754360999999997 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:09.227589+0000", 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6356e-05 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:54.602 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.604 INFO:teuthology.orchestra.run.smithi096.stderr:noscrub is unset 2024-09-22T01:16:54.604 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-22T01:16:54.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-22T01:16:54.726 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:54.726 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:54.726 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.739 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-22T01:16:54.859 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:54.860 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:54.860 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:54.873 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-22T01:16:54.998 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154807 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.77907226, 2024-09-22T01:16:54.999 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024696026999999999, 2024-09-22T01:16:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.001 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154807 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.002 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216165+0000", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9699999999999996e-07 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216173+0000", 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1259999999999998e-06 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.003 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216181+0000", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.7530000000000001e-06 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216187+0000", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7339999999999996e-06 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240739+0000", 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024551975 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.004 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240794+0000", 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5427999999999999e-05 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240861+0000", 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6414000000000005e-05 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.005 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154815 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 53248~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.779042179999998, 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024812520000000001, 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154815 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.006 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216195+0000", 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6899999999999998e-07 2024-09-22T01:16:55.007 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216198+0000", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216204+0000", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9100000000000002e-06 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216209+0000", 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.198e-06 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.008 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240890+0000", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024681333999999999 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.240939+0000", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8812e-05 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241007+0000", 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8332999999999998e-05 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.009 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154824 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.779020949, 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024941471, 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154824 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.010 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216216+0000", 2024-09-22T01:16:55.011 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216219+0000", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3980000000000002e-06 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216225+0000", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3899999999999998e-06 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216231+0000", 2024-09-22T01:16:55.012 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4110000000000004e-06 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241041+0000", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024810187000000001 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241094+0000", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3130000000000001e-05 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241157+0000", 2024-09-22T01:16:55.013 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3559000000000006e-05 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154833 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 98304~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778999685000002, 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025067048000000001, 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.014 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154833 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216237+0000", 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.015 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216238+0000", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5000000000000002e-07 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216240+0000", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.086e-06 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216247+0000", 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9260000000000003e-06 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.016 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216252+0000", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9679999999999997e-06 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241189+0000", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.024937259999999999 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241239+0000", 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9568999999999999e-05 2024-09-22T01:16:55.017 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241304+0000", 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5488999999999996e-05 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154840 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 120832~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778978377999998, 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025194747, 2024-09-22T01:16:55.018 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154840 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.019 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216259+0000", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.96e-07 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216261+0000", 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.0159999999999998e-06 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.020 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216268+0000", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2869999999999999e-06 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216274+0000", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4790000000000004e-06 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241337+0000", 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025063312000000001 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.021 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241391+0000", 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4185999999999999e-05 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241453+0000", 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2070999999999996e-05 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.022 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154848 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 143360~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778956737999998, 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025319292, 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154848 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.023 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216280+0000", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216281+0000", 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8800000000000002e-07 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.024 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216283+0000", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.1799999999999999e-06 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216301+0000", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8425000000000001e-05 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216310+0000", 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8680000000000005e-06 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.025 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241484+0000", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025173614 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241532+0000", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8486999999999998e-05 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241599+0000", 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7130000000000003e-05 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.026 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154856 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 165888~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778911608000001, 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025417760000000001, 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.027 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154856 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216325+0000", 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.028 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216326+0000", 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.6700000000000002e-07 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216329+0000", 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1360000000000001e-06 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216337+0000", 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.6569999999999997e-06 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.029 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216346+0000", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.5060000000000001e-06 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241633+0000", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025286126999999999 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241678+0000", 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5330000000000001e-05 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.030 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241743+0000", 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5136999999999997e-05 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154864 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 188416~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778880583000003, 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025536110000000001, 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.031 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154864 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216356+0000", 2024-09-22T01:16:55.032 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216357+0000", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4700000000000001e-07 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216359+0000", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.379e-06 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216366+0000", 2024-09-22T01:16:55.033 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8580000000000002e-06 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216373+0000", 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6220000000000003e-06 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241775+0000", 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025401862000000001 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.034 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241823+0000", 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8316000000000003e-05 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241892+0000", 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9425999999999996e-05 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154872 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 210944~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778856544999996, 2024-09-22T01:16:55.035 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025662921000000002, 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154872 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.036 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216380+0000", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216381+0000", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0200000000000002e-07 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216383+0000", 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2369999999999999e-06 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.037 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216390+0000", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5590000000000002e-06 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216395+0000", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2179999999999998e-06 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241925+0000", 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025530258 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.038 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.241978+0000", 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3121000000000002e-05 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242043+0000", 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5025999999999997e-05 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.039 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154880 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 233472~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778835280999999, 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025784893999999999, 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154880 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.040 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216402+0000", 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.15e-07 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.041 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216413+0000", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.096e-05 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216419+0000", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4439999999999996e-06 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216427+0000", 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.3429999999999998e-06 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.042 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242074+0000", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025646720000000001 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242121+0000", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7008000000000001e-05 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242187+0000", 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.6004e-05 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.043 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154888 3.b 3:d7293942:::rbd_data.11246357b32e.0000000000000000:33 [read 256000~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778783249, 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025884793999999999, 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154888 2024-09-22T01:16:55.044 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.045 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216454+0000", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2200000000000003e-07 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216457+0000", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.0240000000000002e-06 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216464+0000", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5230000000000001e-06 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.046 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216469+0000", 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9280000000000001e-06 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242220+0000", 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025751120999999998 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242275+0000", 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4741999999999999e-05 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.047 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242338+0000", 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3733999999999997e-05 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154894 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778724277999999, 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025971372, 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.048 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154894 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:55.049 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216513+0000", 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2600000000000001e-07 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216522+0000", 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8049999999999996e-06 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.050 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216530+0000", 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9279999999999993e-06 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216536+0000", 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6359999999999999e-06 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242367+0000", 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025831131 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.051 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242422+0000", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4521999999999997e-05 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242484+0000", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2524e-05 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154902 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778620511, 2024-09-22T01:16:55.052 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.026017392, 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154902 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.053 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216616+0000", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216617+0000", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.8400000000000003e-07 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216620+0000", 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.7630000000000001e-06 2024-09-22T01:16:55.054 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216628+0000", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.1780000000000003e-06 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.216635+0000", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7970000000000003e-06 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242518+0000", 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025882560999999998 2024-09-22T01:16:55.055 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242573+0000", 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5840000000000001e-05 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242634+0000", 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0469e-05 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.056 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154910 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 57344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.778040728999997, 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025566531, 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154910 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.057 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217196+0000", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217197+0000", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.4300000000000002e-07 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.058 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217201+0000", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.0090000000000001e-06 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217209+0000", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.9379999999999992e-06 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217217+0000", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.2239999999999995e-06 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.059 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242664+0000", 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025447335000000001 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242714+0000", 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9802999999999997e-05 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242763+0000", 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8479000000000001e-05 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.060 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154919 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 79872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777920285999997, 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025572342000000001, 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154919 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.061 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217317+0000", 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5400000000000001e-07 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.062 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217320+0000", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4729999999999999e-06 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217327+0000", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7680000000000003e-06 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217333+0000", 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:55.063 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242796+0000", 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025463056000000001 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242842+0000", 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5989000000000003e-05 2024-09-22T01:16:55.064 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242889+0000", 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7071999999999999e-05 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154927 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 102400~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777840286, 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025617728999999999, 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.065 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154927 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.066 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217397+0000", 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5899999999999996e-07 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.067 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217400+0000", 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.3970000000000001e-06 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217406+0000", 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8480000000000003e-06 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217412+0000", 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.7450000000000001e-06 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.068 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242918+0000", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02550585 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.242968+0000", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0309999999999998e-05 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243014+0000", 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6020000000000003e-05 2024-09-22T01:16:55.069 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154935 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 124928~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777376097999998, 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025275829999999999, 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.070 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154935 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:55.071 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217861+0000", 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217863+0000", 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7719999999999999e-06 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.072 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217867+0000", 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.9230000000000004e-06 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217870+0000", 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.1269999999999999e-06 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243047+0000", 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025176925999999999 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.073 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243092+0000", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5395999999999997e-05 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243137+0000", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4218999999999998e-05 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154942 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 147456~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:55.074 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777306523, 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025331715000000001, 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154942 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:55.075 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217930+0000", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217931+0000", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3200000000000005e-07 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217934+0000", 2024-09-22T01:16:55.076 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.768e-06 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217941+0000", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.2949999999999996e-06 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217946+0000", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2290000000000002e-06 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243167+0000", 2024-09-22T01:16:55.077 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025220929999999999 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243216+0000", 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8980000000000002e-05 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243262+0000", 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5980999999999999e-05 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154950 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 169984~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777239774000002, 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025388707, 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154950 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.079 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217997+0000", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.217998+0000", 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.2199999999999999e-07 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.080 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218000+0000", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.464e-06 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218007+0000", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.477e-06 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218012+0000", 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2120000000000002e-06 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243295+0000", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025283379000000002 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243340+0000", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5260999999999999e-05 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243386+0000", 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5491999999999997e-05 2024-09-22T01:16:55.082 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:154958 3.b 3:d1e8f74b:::rbd_data.11246357b32e.0000000000000014:33 [read 192512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 62.777168156000002, 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.025434279000000001, 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.083 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 154958 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.084 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218069+0000", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6699999999999999e-07 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218072+0000", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.424e-06 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218078+0000", 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.7279999999999998e-06 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.085 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.218084+0000", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.395e-06 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243408+0000", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.02532386 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243457+0000", 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9097000000000001e-05 2024-09-22T01:16:55.086 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:52.243503+0000", 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6307999999999998e-05 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.087 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:55.088 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-22T01:16:55.215 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:55.215 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:55.215 INFO:teuthology.orchestra.run.smithi150.stdout: "num_ops": 0 2024-09-22T01:16:55.215 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:55.229 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-22T01:16:55.353 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:55.353 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [], 2024-09-22T01:16:55.353 INFO:teuthology.orchestra.run.smithi150.stdout: "complaint_time": 30, 2024-09-22T01:16:55.353 INFO:teuthology.orchestra.run.smithi150.stdout: "num_blocked_ops": 0 2024-09-22T01:16:55.354 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:55.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-22T01:16:55.495 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-22T01:16:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "size": 20, 2024-09-22T01:16:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 600, 2024-09-22T01:16:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: "ops": [ 2024-09-22T01:16:55.495 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142815 3.b 3:d4192677:::rbd_data.11246c9a159.0000000000000004:34 [read 18006016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984448658999995, 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353443999999999, 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "started", 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142815 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.496 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507581+0000", 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3599999999999999e-07 2024-09-22T01:16:55.497 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507597+0000", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.5503999999999999e-05 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507609+0000", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2155e-05 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507614+0000", 2024-09-22T01:16:55.498 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9529999999999998e-06 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559728+0000", 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052113877000000003 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "started", 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559831+0000", 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103538 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.499 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559934+0000", 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.000103081 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142816 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984390028999997, 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052346966000000002, 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.500 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142816 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.501 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507639+0000", 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507640+0000", 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507643+0000", 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.475e-06 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.502 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507649+0000", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4389999999999997e-06 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507655+0000", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.6799999999999998e-06 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559978+0000", 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052323394000000002 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.503 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.559986+0000", 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.3350000000000007e-06 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142817 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984367148999993, 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052375814999999999, 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.504 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142817 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507662+0000", 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.505 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507663+0000", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4300000000000003e-07 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507665+0000", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7540000000000001e-06 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507670+0000", 2024-09-22T01:16:55.506 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.5759999999999996e-06 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507675+0000", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4900000000000002e-06 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560025+0000", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350436 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560038+0000", 2024-09-22T01:16:55.507 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2916e-05 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142818 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984349085999995, 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052380616999999997, 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.508 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142818 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507680+0000", 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.509 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507681+0000", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.34e-07 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507682+0000", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.722e-06 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507688+0000", 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1479999999999997e-06 2024-09-22T01:16:55.510 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507710+0000", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.2493999999999998e-05 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560054+0000", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052344228 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560061+0000", 2024-09-22T01:16:55.511 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.691e-06 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142819 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984311598999994, 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052396824000000002, 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.512 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142819 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.513 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507718+0000", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.7599999999999998e-07 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507722+0000", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.5619999999999999e-06 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507731+0000", 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.8969999999999997e-06 2024-09-22T01:16:55.514 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507736+0000", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8690000000000003e-06 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560101+0000", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365041000000001 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560115+0000", 2024-09-22T01:16:55.515 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4079e-05 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142820 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984285976999999, 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052408105000000003, 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.516 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142820 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.517 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507744+0000", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.5099999999999999e-07 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507746+0000", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8729999999999999e-06 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507754+0000", 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 7.5739999999999998e-06 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.518 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507759+0000", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9479999999999999e-06 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560145+0000", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052386174000000001 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560152+0000", 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8850000000000001e-06 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.519 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142821 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984265128999994, 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052438163000000003, 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.520 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142821 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507764+0000", 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.521 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507765+0000", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3799999999999998e-07 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507767+0000", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8500000000000001e-06 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507772+0000", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3229999999999997e-06 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.522 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507776+0000", 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4050000000000002e-06 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560191+0000", 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052414296999999999 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560203+0000", 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1950000000000001e-05 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.523 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142822 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984247890000006, 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052441785999999997, 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142822 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.524 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507782+0000", 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.4200000000000002e-07 2024-09-22T01:16:55.525 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507784+0000", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7889999999999999e-06 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507789+0000", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.0300000000000001e-06 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507793+0000", 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.408e-06 2024-09-22T01:16:55.526 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560217+0000", 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423850000000001 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560223+0000", 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3670000000000002e-06 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.527 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142823 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984195795999995, 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440209000000002, 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142823 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.528 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507834+0000", 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4899999999999995e-07 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.529 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507837+0000", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.6840000000000001e-06 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507843+0000", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1380000000000004e-06 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507848+0000", 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.5410000000000002e-06 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.530 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560262+0000", 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413992 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560274+0000", 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.2305e-05 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142824 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.531 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984175922999995, 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052440946000000002, 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142824 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.532 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507854+0000", 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3200000000000001e-07 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.533 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507856+0000", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.657e-06 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507861+0000", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1540000000000001e-06 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507865+0000", 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.3170000000000003e-06 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.534 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560288+0000", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052423175000000002 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560295+0000", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3110000000000002e-06 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142825 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:55.535 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984129675000005, 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052429569000000002, 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142825 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.536 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507900+0000", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.1799999999999995e-07 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507902+0000", 2024-09-22T01:16:55.537 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.967e-06 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507908+0000", 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8930000000000002e-06 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507913+0000", 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.538 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560318+0000", 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052404907000000001 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560329+0000", 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1725999999999999e-05 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142826 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984108988000003, 2024-09-22T01:16:55.539 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052430255000000002, 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142826 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.540 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507921+0000", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3700000000000001e-07 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507923+0000", 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.7680000000000001e-06 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.541 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507928+0000", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4469999999999997e-06 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507933+0000", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4580000000000003e-06 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560344+0000", 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052411803999999999 2024-09-22T01:16:55.542 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560351+0000", 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4409999999999998e-06 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142827 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.984060299000006, 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052413973000000003, 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.543 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142827 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.544 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507969+0000", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507970+0000", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507984+0000", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.4445e-05 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.545 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.507995+0000", 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1034e-05 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508000+0000", 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.8720000000000001e-06 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560371+0000", 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052371305 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.546 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560383+0000", 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1688999999999999e-05 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142828 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983991246000002, 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052365536999999997, 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.547 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142828 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508038+0000", 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.548 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508039+0000", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2799999999999996e-07 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508041+0000", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.4269999999999998e-06 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508048+0000", 2024-09-22T01:16:55.549 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.4479999999999999e-06 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508052+0000", 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4569999999999998e-06 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560397+0000", 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052345246999999998 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.550 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560404+0000", 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.3300000000000004e-06 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142829 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983971412000002, 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052378952999999999, 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.551 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142829 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508058+0000", 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.552 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508059+0000", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2399999999999998e-07 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508060+0000", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.745e-06 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508066+0000", 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.1939999999999997e-06 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.553 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508072+0000", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.0560000000000001e-06 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560425+0000", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353644999999997 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560437+0000", 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1789e-05 2024-09-22T01:16:55.554 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142830 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983922781000004, 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052350671000000001, 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.555 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142830 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.556 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508107+0000", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.2300000000000001e-07 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508109+0000", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 2.007e-06 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508115+0000", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.9780000000000002e-06 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508120+0000", 2024-09-22T01:16:55.557 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.091e-06 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560451+0000", 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052330817000000002 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560457+0000", 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.2550000000000003e-06 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.558 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142831 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983901720999995, 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052376945000000001, 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142831 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.559 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508128+0000", 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 3.3099999999999999e-07 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.560 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508130+0000", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.714e-06 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508135+0000", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.3619999999999997e-06 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508140+0000", 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4329999999999997e-06 2024-09-22T01:16:55.561 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560493+0000", 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052353655999999998 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560505+0000", 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1449e-05 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.562 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142832 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983851157999993, 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052348324000000002, 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142832 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.563 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508178+0000", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508179+0000", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.4499999999999997e-07 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.564 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508181+0000", 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.8649999999999999e-06 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508187+0000", 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.8860000000000001e-06 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508191+0000", 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.4279999999999998e-06 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.565 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560519+0000", 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052327599000000002 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560527+0000", 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 8.0010000000000001e-06 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142833 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.566 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983830230999999, 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052359409000000003, 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142833 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.567 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508199+0000", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508209+0000", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 9.6139999999999998e-06 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.568 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508211+0000", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.9589999999999998e-06 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508216+0000", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 5.468e-06 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508221+0000", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.6779999999999999e-06 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.569 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560547+0000", 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052325773999999999 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560559+0000", 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 1.1916000000000001e-05 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "description": "osd_op(client.4388.0:142834 3.b 3.ee64982b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e108)", 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "initiated_at": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:55.570 INFO:teuthology.orchestra.run.smithi150.stdout: "age": 68.983802343999997, 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.052368565999999998, 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "continuous": false, 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "type_data": { 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "flag_point": "reached pg", 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_info": { 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client": "client.4388", 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "client_addr": "172.21.15.96:0/3826584466", 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "tid": 142834 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "events": [ 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "initiated", 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.571 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "header_read", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.508227+0000", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "throttled", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511401+0000", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.0031738959999999998 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "all_read", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511406+0000", 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.7779999999999998e-06 2024-09-22T01:16:55.572 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "dispatched", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511413+0000", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.9909999999999996e-06 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "queued_for_pg", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.511418+0000", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 4.9300000000000002e-06 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "reached_pg", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560589+0000", 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 0.049171104 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-09-22T01:16:55.573 INFO:teuthology.orchestra.run.smithi150.stdout: { 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "event": "done", 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "time": "2024-09-22T01:15:46.560596+0000", 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: "duration": 6.8669999999999996e-06 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-22T01:16:55.574 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-22T01:16:55.689 INFO:teuthology.orchestra.run.smithi096.stderr:nodeep-scrub is unset 2024-09-22T01:16:58.700 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-22T01:16:58.844 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a: {} 2024-09-22T01:16:58.844 INFO:teuthology.orchestra.run.smithi096.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:16:58.857 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b: {} 2024-09-22T01:16:58.857 INFO:teuthology.orchestra.run.smithi096.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:16:58.867 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c: {} 2024-09-22T01:16:58.867 INFO:teuthology.orchestra.run.smithi096.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-22T01:16:58.885 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-22T01:16:59.535 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.0 to 1 (10000) 2024-09-22T01:16:59.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-22T01:17:00.715 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-22T01:17:00.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-22T01:17:01.710 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.1 to 1 (10000) 2024-09-22T01:17:01.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-22T01:17:02.732 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-22T01:17:02.759 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-22T01:17:03.744 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.2 to 1 (10000) 2024-09-22T01:17:03.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-22T01:17:04.537 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-22T01:17:04.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-22T01:17:05.754 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.3 to 1 (10000) 2024-09-22T01:17:05.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-22T01:17:06.765 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-22T01:17:06.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-22T01:17:07.775 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.4 to 1 (10000) 2024-09-22T01:17:07.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-22T01:17:08.779 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-22T01:17:08.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-22T01:17:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.5 to 1 (10000) 2024-09-22T01:17:09.829 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-22T01:17:10.807 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-22T01:17:10.830 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-22T01:17:11.825 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.6 to 1 (10000) 2024-09-22T01:17:11.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-22T01:17:12.827 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-22T01:17:12.853 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-22T01:17:13.843 INFO:teuthology.orchestra.run.smithi096.stderr:reweighted osd.7 to 1 (10000) 2024-09-22T01:17:13.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-22T01:17:14.548 INFO:teuthology.orchestra.run.smithi096.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-22T01:17:14.576 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-22T01:17:14.577 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-22T01:17:14.577 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-22T01:17:14.577 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-22T01:17:14.577 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-22T01:17:14.577 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:17:14.880 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:14.880 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":191,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:17:14.536938+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:14:37.510453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:13:53.081727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:14:03.263074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[4,7]},{"pgid":"3.16","osds":[2,0]}],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.0","osds":[5]},{"pgid":"3.6","osds":[4]},{"pgid":"3.8","osds":[4]},{"pgid":"3.b","osds":[0,6]},{"pgid":"3.c","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:17:14.899 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-22T01:17:14.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:17:15.196 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:15.196 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":191,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:17:14.536938+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:14:37.510453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:13:53.081727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:14:03.263074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[4,7]},{"pgid":"3.16","osds":[2,0]}],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.0","osds":[5]},{"pgid":"3.6","osds":[4]},{"pgid":"3.8","osds":[4]},{"pgid":"3.b","osds":[0,6]},{"pgid":"3.c","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:17:15.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-22T01:17:15.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-22T01:17:15.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-22T01:17:15.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-22T01:17:15.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-22T01:17:15.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-22T01:17:15.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-22T01:17:15.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-22T01:17:15.456 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:17:15.460 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:17:15.471 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.472 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:17:15.484 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:15.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:17:15.486 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.486 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:17:15.487 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:17:15.489 INFO:teuthology.orchestra.run.smithi096.stdout:60129542190 2024-09-22T01:17:15.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:17:15.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:17:15.618+0000 7f2fdeffd640 0 --2- 172.21.15.96:0/1463480839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fd800a470 0x7f2fd80c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-22T01:17:15.694 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:15.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:17:15.937 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:15.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.6 2024-09-22T01:17:15.972 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:15.987 INFO:teuthology.orchestra.run.smithi096.stdout:64424509483 2024-09-22T01:17:16.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.1 2024-09-22T01:17:16.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509483 for osd.5 2024-09-22T01:17:16.017 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:16.039 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:16.041 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:16.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.7 2024-09-22T01:17:16.057 INFO:teuthology.orchestra.run.smithi096.stdout:60129542189 2024-09-22T01:17:16.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.4 2024-09-22T01:17:16.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.2 2024-09-22T01:17:16.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542189 for osd.0 2024-09-22T01:17:16.135 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:16.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509485 for osd.3 2024-09-22T01:17:16.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-22T01:17:17.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-22T01:17:17.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-22T01:17:17.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-22T01:17:17.063 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-22T01:17:17.068 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-22T01:17:17.077 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-22T01:17:17.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:17:17.461 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:17.484 INFO:teuthology.orchestra.run.smithi096.stdout:64424509485 2024-09-22T01:17:17.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.6 2024-09-22T01:17:17.485 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.510 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509485 got 64424509485 for osd.5 2024-09-22T01:17:17.510 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:17.510 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.1 2024-09-22T01:17:17.527 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.536 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:17.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.7 2024-09-22T01:17:17.559 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.582 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:17.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.4 2024-09-22T01:17:17.605 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.614 INFO:teuthology.orchestra.run.smithi096.stdout:60129542190 2024-09-22T01:17:17.620 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:17.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.2 2024-09-22T01:17:17.642 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542190 got 60129542190 for osd.0 2024-09-22T01:17:17.643 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:17.653 INFO:teuthology.orchestra.run.smithi096.stdout:64424509486 2024-09-22T01:17:17.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509486 for osd.3 2024-09-22T01:17:18.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-22T01:17:18.968 INFO:teuthology.orchestra.run.smithi096.stdout:64424509487 2024-09-22T01:17:18.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509487 got 64424509487 for osd.3 2024-09-22T01:17:18.987 DEBUG:teuthology.parallel:result is None 2024-09-22T01:17:18.987 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-22T01:17:18.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:19.293 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:19.294 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:19.312 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-22T01:17:18.555506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1456,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":414388,"kb_used_data":19140,"kb_used_omap":12,"kb_used_meta":395059,"kb_avail":754560332,"statfs":{"total":773094113280,"available":772669779968,"internally_reserved":0,"allocated":19599360,"data_stored":10040233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":404540976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.716902"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3313,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.552271+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:14.552271+0000","last_peered":"2024-09-22T01:17:14.552271+0000","last_clean":"2024-09-22T01:17:14.552271+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:14.552271+0000","last_undegraded":"2024-09-22T01:17:14.552271+0000","last_fullsized":"2024-09-22T01:17:14.552271+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12822,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.552085+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:14.552085+0000","last_peered":"2024-09-22T01:17:14.552085+0000","last_clean":"2024-09-22T01:17:14.552085+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:14.552085+0000","last_undegraded":"2024-09-22T01:17:14.552085+0000","last_fullsized":"2024-09-22T01:17:14.552085+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20546,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.549005+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:15.549005+0000","last_peered":"2024-09-22T01:17:15.549005+0000","last_clean":"2024-09-22T01:17:15.549005+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:15.549005+0000","last_undegraded":"2024-09-22T01:17:15.549005+0000","last_fullsized":"2024-09-22T01:17:15.549005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13027,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.551729+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:14.551729+0000","last_peered":"2024-09-22T01:17:14.551729+0000","last_clean":"2024-09-22T01:17:14.551729+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:14.551729+0000","last_undegraded":"2024-09-22T01:17:14.551729+0000","last_fullsized":"2024-09-22T01:17:14.551729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5861,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.544667+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:14.544667+0000","last_peered":"2024-09-22T01:17:14.544667+0000","last_clean":"2024-09-22T01:17:14.544667+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:14.544667+0000","last_undegraded":"2024-09-22T01:17:14.544667+0000","last_fullsized":"2024-09-22T01:17:14.544667+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17413,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.552162+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:14.552162+0000","last_peered":"2024-09-22T01:17:14.552162+0000","last_clean":"2024-09-22T01:17:14.552162+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:14.552162+0000","last_undegraded":"2024-09-22T01:17:14.552162+0000","last_fullsized":"2024-09-22T01:17:14.552162+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6604,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.544740+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:14.544740+0000","last_peered":"2024-09-22T01:17:14.544740+0000","last_clean":"2024-09-22T01:17:14.544740+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:14.544740+0000","last_undegraded":"2024-09-22T01:17:14.544740+0000","last_fullsized":"2024-09-22T01:17:14.544740+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6638,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.553759+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:14.553759+0000","last_peered":"2024-09-22T01:17:14.553759+0000","last_clean":"2024-09-22T01:17:14.553759+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:14.553759+0000","last_undegraded":"2024-09-22T01:17:14.553759+0000","last_fullsized":"2024-09-22T01:17:14.553759+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5794,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.552497+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:14.552497+0000","last_peered":"2024-09-22T01:17:14.552497+0000","last_clean":"2024-09-22T01:17:14.552497+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:14.552497+0000","last_undegraded":"2024-09-22T01:17:14.552497+0000","last_fullsized":"2024-09-22T01:17:14.552497+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13501,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.546150+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:14.546150+0000","last_peered":"2024-09-22T01:17:14.546150+0000","last_clean":"2024-09-22T01:17:14.546150+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:14.546150+0000","last_undegraded":"2024-09-22T01:17:14.546150+0000","last_fullsized":"2024-09-22T01:17:14.546150+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549410+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:17.549410+0000","last_peered":"2024-09-22T01:17:17.549410+0000","last_clean":"2024-09-22T01:17:17.549410+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:17.549410+0000","last_undegraded":"2024-09-22T01:17:17.549410+0000","last_fullsized":"2024-09-22T01:17:17.549410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10655,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548912+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:15.548912+0000","last_peered":"2024-09-22T01:17:15.548912+0000","last_clean":"2024-09-22T01:17:15.548912+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:15.548912+0000","last_undegraded":"2024-09-22T01:17:15.548912+0000","last_fullsized":"2024-09-22T01:17:15.548912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":335,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.549083+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:15.549083+0000","last_peered":"2024-09-22T01:17:15.549083+0000","last_clean":"2024-09-22T01:17:15.549083+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:15.549083+0000","last_undegraded":"2024-09-22T01:17:15.549083+0000","last_fullsized":"2024-09-22T01:17:15.549083+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4702,"reported_epoch":190,"state":"remapped+peering","last_fresh":"2024-09-22T01:17:13.855938+0000","last_change":"2024-09-22T01:17:14.546080+0000","last_active":"2024-09-22T01:17:13.855938+0000","last_peered":"2024-09-22T01:17:12.827374+0000","last_clean":"2024-09-22T01:17:12.827374+0000","last_became_active":"2024-09-22T01:15:59.611357+0000","last_became_peered":"2024-09-22T01:15:59.611357+0000","last_unstale":"2024-09-22T01:17:13.855938+0000","last_undegraded":"2024-09-22T01:17:13.855938+0000","last_fullsized":"2024-09-22T01:17:13.855938+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":183,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.544881+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:14.544881+0000","last_peered":"2024-09-22T01:17:14.544881+0000","last_clean":"2024-09-22T01:17:14.544881+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:14.544881+0000","last_undegraded":"2024-09-22T01:17:14.544881+0000","last_fullsized":"2024-09-22T01:17:14.544881+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4315,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.546599+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:14.546599+0000","last_peered":"2024-09-22T01:17:14.546599+0000","last_clean":"2024-09-22T01:17:14.546599+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:14.546599+0000","last_undegraded":"2024-09-22T01:17:14.546599+0000","last_fullsized":"2024-09-22T01:17:14.546599+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":335,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.549177+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:15.549177+0000","last_peered":"2024-09-22T01:17:15.549177+0000","last_clean":"2024-09-22T01:17:15.549177+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:15.549177+0000","last_undegraded":"2024-09-22T01:17:15.549177+0000","last_fullsized":"2024-09-22T01:17:15.549177+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3686,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.545795+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:14.545795+0000","last_peered":"2024-09-22T01:17:14.545795+0000","last_clean":"2024-09-22T01:17:14.545795+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:14.545795+0000","last_undegraded":"2024-09-22T01:17:14.545795+0000","last_fullsized":"2024-09-22T01:17:14.545795+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":352,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.552078+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:14.552078+0000","last_peered":"2024-09-22T01:17:14.552078+0000","last_clean":"2024-09-22T01:17:14.552078+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:14.552078+0000","last_undegraded":"2024-09-22T01:17:14.552078+0000","last_fullsized":"2024-09-22T01:17:14.552078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8753,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.546405+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:14.546405+0000","last_peered":"2024-09-22T01:17:14.546405+0000","last_clean":"2024-09-22T01:17:14.546405+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:14.546405+0000","last_undegraded":"2024-09-22T01:17:14.546405+0000","last_fullsized":"2024-09-22T01:17:14.546405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6978,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.550623+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:17.550623+0000","last_peered":"2024-09-22T01:17:17.550623+0000","last_clean":"2024-09-22T01:17:17.550623+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:17.550623+0000","last_undegraded":"2024-09-22T01:17:17.550623+0000","last_fullsized":"2024-09-22T01:17:17.550623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":339,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548984+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:15.548984+0000","last_peered":"2024-09-22T01:17:15.548984+0000","last_clean":"2024-09-22T01:17:15.548984+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:15.548984+0000","last_undegraded":"2024-09-22T01:17:15.548984+0000","last_fullsized":"2024-09-22T01:17:15.548984+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4321,"reported_epoch":190,"state":"remapped+peering","last_fresh":"2024-09-22T01:17:13.847221+0000","last_change":"2024-09-22T01:17:14.553084+0000","last_active":"2024-09-22T01:17:13.847221+0000","last_peered":"2024-09-22T01:17:12.838537+0000","last_clean":"2024-09-22T01:17:12.838537+0000","last_became_active":"2024-09-22T01:15:51.529181+0000","last_became_peered":"2024-09-22T01:15:51.529181+0000","last_unstale":"2024-09-22T01:17:13.847221+0000","last_undegraded":"2024-09-22T01:17:13.847221+0000","last_fullsized":"2024-09-22T01:17:13.847221+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.556710+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:14.556710+0000","last_peered":"2024-09-22T01:17:14.556710+0000","last_clean":"2024-09-22T01:17:14.556710+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:14.556710+0000","last_undegraded":"2024-09-22T01:17:14.556710+0000","last_fullsized":"2024-09-22T01:17:14.556710+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7028,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.544569+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:14.544569+0000","last_peered":"2024-09-22T01:17:14.544569+0000","last_clean":"2024-09-22T01:17:14.544569+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:14.544569+0000","last_undegraded":"2024-09-22T01:17:14.544569+0000","last_fullsized":"2024-09-22T01:17:14.544569+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":335,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548714+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:15.548714+0000","last_peered":"2024-09-22T01:17:15.548714+0000","last_clean":"2024-09-22T01:17:15.548714+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:15.548714+0000","last_undegraded":"2024-09-22T01:17:15.548714+0000","last_fullsized":"2024-09-22T01:17:15.548714+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6570,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.544533+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:14.544533+0000","last_peered":"2024-09-22T01:17:14.544533+0000","last_clean":"2024-09-22T01:17:14.544533+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:14.544533+0000","last_undegraded":"2024-09-22T01:17:14.544533+0000","last_fullsized":"2024-09-22T01:17:14.544533+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":357,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.549080+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:15.549080+0000","last_peered":"2024-09-22T01:17:15.549080+0000","last_clean":"2024-09-22T01:17:15.549080+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:15.549080+0000","last_undegraded":"2024-09-22T01:17:15.549080+0000","last_fullsized":"2024-09-22T01:17:15.549080+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":505,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.551940+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:14.551940+0000","last_peered":"2024-09-22T01:17:14.551940+0000","last_clean":"2024-09-22T01:17:14.551940+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:14.551940+0000","last_undegraded":"2024-09-22T01:17:14.551940+0000","last_fullsized":"2024-09-22T01:17:14.551940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4000,"reported_epoch":190,"state":"remapped+peering","last_fresh":"2024-09-22T01:17:13.853644+0000","last_change":"2024-09-22T01:17:14.545753+0000","last_active":"2024-09-22T01:17:13.853644+0000","last_peered":"2024-09-22T01:17:13.585279+0000","last_clean":"2024-09-22T01:17:13.585279+0000","last_became_active":"2024-09-22T01:17:09.807767+0000","last_became_peered":"2024-09-22T01:17:09.807767+0000","last_unstale":"2024-09-22T01:17:13.853644+0000","last_undegraded":"2024-09-22T01:17:13.853644+0000","last_fullsized":"2024-09-22T01:17:13.853644+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19409,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548890+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:15.548890+0000","last_peered":"2024-09-22T01:17:15.548890+0000","last_clean":"2024-09-22T01:17:15.548890+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:15.548890+0000","last_undegraded":"2024-09-22T01:17:15.548890+0000","last_fullsized":"2024-09-22T01:17:15.548890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4578,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548891+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:15.548891+0000","last_peered":"2024-09-22T01:17:15.548891+0000","last_clean":"2024-09-22T01:17:15.548891+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:15.548891+0000","last_undegraded":"2024-09-22T01:17:15.548891+0000","last_fullsized":"2024-09-22T01:17:15.548891+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4994,"reported_epoch":193,"state":"active+recovering+remapped","last_fresh":"2024-09-22T01:17:17.582362+0000","last_change":"2024-09-22T01:17:17.532983+0000","last_active":"2024-09-22T01:17:17.582362+0000","last_peered":"2024-09-22T01:17:17.582362+0000","last_clean":"2024-09-22T01:17:11.824095+0000","last_became_active":"2024-09-22T01:17:15.893174+0000","last_became_peered":"2024-09-22T01:17:15.893174+0000","last_unstale":"2024-09-22T01:17:17.582362+0000","last_undegraded":"2024-09-22T01:17:17.582362+0000","last_fullsized":"2024-09-22T01:17:17.582362+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[{"shards":"0,6","objects":11}],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6316,"reported_epoch":192,"state":"peering","last_fresh":"2024-09-22T01:17:16.554559+0000","last_change":"2024-09-22T01:17:17.549707+0000","last_active":"2024-09-22T01:17:17.549707+0000","last_peered":"2024-09-22T01:17:16.554559+0000","last_clean":"2024-09-22T01:17:12.838920+0000","last_became_active":"2024-09-22T01:17:15.877839+0000","last_became_peered":"2024-09-22T01:17:15.877839+0000","last_unstale":"2024-09-22T01:17:16.554559+0000","last_undegraded":"2024-09-22T01:17:16.554559+0000","last_fullsized":"2024-09-22T01:17:15.553269+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3045,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.549091+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:15.549091+0000","last_peered":"2024-09-22T01:17:15.549091+0000","last_clean":"2024-09-22T01:17:15.549091+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:15.549091+0000","last_undegraded":"2024-09-22T01:17:15.549091+0000","last_fullsized":"2024-09-22T01:17:15.549091+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4143,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.546511+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:14.546511+0000","last_peered":"2024-09-22T01:17:14.546511+0000","last_clean":"2024-09-22T01:17:14.546511+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:14.546511+0000","last_undegraded":"2024-09-22T01:17:14.546511+0000","last_fullsized":"2024-09-22T01:17:14.546511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4099,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549705+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:17.549705+0000","last_peered":"2024-09-22T01:17:17.549705+0000","last_clean":"2024-09-22T01:17:17.549705+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:17.549705+0000","last_undegraded":"2024-09-22T01:17:17.549705+0000","last_fullsized":"2024-09-22T01:17:17.549705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3847,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.551720+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:14.551720+0000","last_peered":"2024-09-22T01:17:14.551720+0000","last_clean":"2024-09-22T01:17:14.551720+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:14.551720+0000","last_undegraded":"2024-09-22T01:17:14.551720+0000","last_fullsized":"2024-09-22T01:17:14.551720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5314,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548800+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:15.548800+0000","last_peered":"2024-09-22T01:17:15.548800+0000","last_clean":"2024-09-22T01:17:15.548800+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:15.548800+0000","last_undegraded":"2024-09-22T01:17:15.548800+0000","last_fullsized":"2024-09-22T01:17:15.548800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5310,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-22T01:17:14.546068+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:14.546068+0000","last_peered":"2024-09-22T01:17:14.546068+0000","last_clean":"2024-09-22T01:17:14.546068+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:14.546068+0000","last_undegraded":"2024-09-22T01:17:14.546068+0000","last_fullsized":"2024-09-22T01:17:14.546068+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6040,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-09-22T01:17:15.548990+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:15.548990+0000","last_peered":"2024-09-22T01:17:15.548990+0000","last_clean":"2024-09-22T01:17:15.548990+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:15.548990+0000","last_undegraded":"2024-09-22T01:17:15.548990+0000","last_fullsized":"2024-09-22T01:17:15.548990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1451,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41752,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330088,"statfs":{"total":96636764160,"available":96594010112,"internally_reserved":0,"allocated":2297856,"data_stored":1106672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2301952,"data_stored":1106691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2895872,"data_stored":1697059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2297856,"data_stored":1106672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59808,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312032,"statfs":{"total":96636764160,"available":96575520768,"internally_reserved":0,"allocated":2301952,"data_stored":1106691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509487,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59800,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312040,"statfs":{"total":96636764160,"available":96575528960,"internally_reserved":0,"allocated":2297856,"data_stored":1106672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56224,"kb_used_data":2828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315616,"statfs":{"total":96636764160,"available":96579190784,"internally_reserved":0,"allocated":2895872,"data_stored":1697059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2310144,"data_stored":1112717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:19.314 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:19.618 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:19.619 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:19.637 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-22T01:17:19.583495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402056,"kb_used_data":19296,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572664,"statfs":{"total":773094113280,"available":772682407936,"internally_reserved":0,"allocated":19759104,"data_stored":10118746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040487"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3319,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562815+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:18.562815+0000","last_peered":"2024-09-22T01:17:18.562815+0000","last_clean":"2024-09-22T01:17:18.562815+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:18.562815+0000","last_undegraded":"2024-09-22T01:17:18.562815+0000","last_fullsized":"2024-09-22T01:17:18.562815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12828,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562766+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:18.562766+0000","last_peered":"2024-09-22T01:17:18.562766+0000","last_clean":"2024-09-22T01:17:18.562766+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:18.562766+0000","last_undegraded":"2024-09-22T01:17:18.562766+0000","last_fullsized":"2024-09-22T01:17:18.562766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20548,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553190+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:17.553190+0000","last_peered":"2024-09-22T01:17:17.553190+0000","last_clean":"2024-09-22T01:17:17.553190+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:17.553190+0000","last_undegraded":"2024-09-22T01:17:17.553190+0000","last_fullsized":"2024-09-22T01:17:17.553190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13033,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.569031+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:18.569031+0000","last_peered":"2024-09-22T01:17:18.569031+0000","last_clean":"2024-09-22T01:17:18.569031+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:18.569031+0000","last_undegraded":"2024-09-22T01:17:18.569031+0000","last_fullsized":"2024-09-22T01:17:18.569031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5867,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564341+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:18.564341+0000","last_peered":"2024-09-22T01:17:18.564341+0000","last_clean":"2024-09-22T01:17:18.564341+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:18.564341+0000","last_undegraded":"2024-09-22T01:17:18.564341+0000","last_fullsized":"2024-09-22T01:17:18.564341+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17419,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568835+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:18.568835+0000","last_peered":"2024-09-22T01:17:18.568835+0000","last_clean":"2024-09-22T01:17:18.568835+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:18.568835+0000","last_undegraded":"2024-09-22T01:17:18.568835+0000","last_fullsized":"2024-09-22T01:17:18.568835+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6610,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563898+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:18.563898+0000","last_peered":"2024-09-22T01:17:18.563898+0000","last_clean":"2024-09-22T01:17:18.563898+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:18.563898+0000","last_undegraded":"2024-09-22T01:17:18.563898+0000","last_fullsized":"2024-09-22T01:17:18.563898+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564236+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:18.564236+0000","last_peered":"2024-09-22T01:17:18.564236+0000","last_clean":"2024-09-22T01:17:18.564236+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:18.564236+0000","last_undegraded":"2024-09-22T01:17:18.564236+0000","last_fullsized":"2024-09-22T01:17:18.564236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5800,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562939+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:18.562939+0000","last_peered":"2024-09-22T01:17:18.562939+0000","last_clean":"2024-09-22T01:17:18.562939+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:18.562939+0000","last_undegraded":"2024-09-22T01:17:18.562939+0000","last_fullsized":"2024-09-22T01:17:18.562939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13507,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.559668+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:18.559668+0000","last_peered":"2024-09-22T01:17:18.559668+0000","last_clean":"2024-09-22T01:17:18.559668+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:18.559668+0000","last_undegraded":"2024-09-22T01:17:18.559668+0000","last_fullsized":"2024-09-22T01:17:18.559668+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549410+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:17.549410+0000","last_peered":"2024-09-22T01:17:17.549410+0000","last_clean":"2024-09-22T01:17:17.549410+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:17.549410+0000","last_undegraded":"2024-09-22T01:17:17.549410+0000","last_fullsized":"2024-09-22T01:17:17.549410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10657,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553351+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:17.553351+0000","last_peered":"2024-09-22T01:17:17.553351+0000","last_clean":"2024-09-22T01:17:17.553351+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:17.553351+0000","last_undegraded":"2024-09-22T01:17:17.553351+0000","last_fullsized":"2024-09-22T01:17:17.553351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552665+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:17.552665+0000","last_peered":"2024-09-22T01:17:17.552665+0000","last_clean":"2024-09-22T01:17:17.552665+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:17.552665+0000","last_undegraded":"2024-09-22T01:17:17.552665+0000","last_fullsized":"2024-09-22T01:17:17.552665+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4951,"reported_epoch":193,"state":"peering","last_fresh":"2024-09-22T01:17:18.484091+0000","last_change":"2024-09-22T01:17:18.565060+0000","last_active":"2024-09-22T01:17:18.565060+0000","last_peered":"2024-09-22T01:17:18.484091+0000","last_clean":"2024-09-22T01:17:12.827374+0000","last_became_active":"2024-09-22T01:17:15.893050+0000","last_became_peered":"2024-09-22T01:17:15.893050+0000","last_unstale":"2024-09-22T01:17:18.484091+0000","last_undegraded":"2024-09-22T01:17:18.484091+0000","last_fullsized":"2024-09-22T01:17:15.546221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564276+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:18.564276+0000","last_peered":"2024-09-22T01:17:18.564276+0000","last_clean":"2024-09-22T01:17:18.564276+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:18.564276+0000","last_undegraded":"2024-09-22T01:17:18.564276+0000","last_fullsized":"2024-09-22T01:17:18.564276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564638+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:18.564638+0000","last_peered":"2024-09-22T01:17:18.564638+0000","last_clean":"2024-09-22T01:17:18.564638+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:18.564638+0000","last_undegraded":"2024-09-22T01:17:18.564638+0000","last_fullsized":"2024-09-22T01:17:18.564638+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553279+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:17.553279+0000","last_peered":"2024-09-22T01:17:17.553279+0000","last_clean":"2024-09-22T01:17:17.553279+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:17.553279+0000","last_undegraded":"2024-09-22T01:17:17.553279+0000","last_fullsized":"2024-09-22T01:17:17.553279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3692,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558357+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:18.558357+0000","last_peered":"2024-09-22T01:17:18.558357+0000","last_clean":"2024-09-22T01:17:18.558357+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:18.558357+0000","last_undegraded":"2024-09-22T01:17:18.558357+0000","last_fullsized":"2024-09-22T01:17:18.558357+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":358,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562708+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:18.562708+0000","last_peered":"2024-09-22T01:17:18.562708+0000","last_clean":"2024-09-22T01:17:18.562708+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:18.562708+0000","last_undegraded":"2024-09-22T01:17:18.562708+0000","last_fullsized":"2024-09-22T01:17:18.562708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8759,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560762+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:18.560762+0000","last_peered":"2024-09-22T01:17:18.560762+0000","last_clean":"2024-09-22T01:17:18.560762+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:18.560762+0000","last_undegraded":"2024-09-22T01:17:18.560762+0000","last_fullsized":"2024-09-22T01:17:18.560762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6978,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.550623+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:17.550623+0000","last_peered":"2024-09-22T01:17:17.550623+0000","last_clean":"2024-09-22T01:17:17.550623+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:17.550623+0000","last_undegraded":"2024-09-22T01:17:17.550623+0000","last_fullsized":"2024-09-22T01:17:17.550623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553031+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:17.553031+0000","last_peered":"2024-09-22T01:17:17.553031+0000","last_clean":"2024-09-22T01:17:17.553031+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:17.553031+0000","last_undegraded":"2024-09-22T01:17:17.553031+0000","last_fullsized":"2024-09-22T01:17:17.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4604,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.573188+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:18.573188+0000","last_peered":"2024-09-22T01:17:18.573188+0000","last_clean":"2024-09-22T01:17:18.573188+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:18.573188+0000","last_undegraded":"2024-09-22T01:17:18.573188+0000","last_fullsized":"2024-09-22T01:17:18.573188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558136+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:18.558136+0000","last_peered":"2024-09-22T01:17:18.558136+0000","last_clean":"2024-09-22T01:17:18.558136+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:18.558136+0000","last_undegraded":"2024-09-22T01:17:18.558136+0000","last_fullsized":"2024-09-22T01:17:18.558136+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563777+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:18.563777+0000","last_peered":"2024-09-22T01:17:18.563777+0000","last_clean":"2024-09-22T01:17:18.563777+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:18.563777+0000","last_undegraded":"2024-09-22T01:17:18.563777+0000","last_fullsized":"2024-09-22T01:17:18.563777+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553473+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:17.553473+0000","last_peered":"2024-09-22T01:17:17.553473+0000","last_clean":"2024-09-22T01:17:17.553473+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:17.553473+0000","last_undegraded":"2024-09-22T01:17:17.553473+0000","last_fullsized":"2024-09-22T01:17:17.553473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6576,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564025+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:18.564025+0000","last_peered":"2024-09-22T01:17:18.564025+0000","last_clean":"2024-09-22T01:17:18.564025+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:18.564025+0000","last_undegraded":"2024-09-22T01:17:18.564025+0000","last_fullsized":"2024-09-22T01:17:18.564025+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":359,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553135+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:17.553135+0000","last_peered":"2024-09-22T01:17:17.553135+0000","last_clean":"2024-09-22T01:17:17.553135+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:17.553135+0000","last_undegraded":"2024-09-22T01:17:17.553135+0000","last_fullsized":"2024-09-22T01:17:17.553135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":511,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568759+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:18.568759+0000","last_peered":"2024-09-22T01:17:18.568759+0000","last_clean":"2024-09-22T01:17:18.568759+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:18.568759+0000","last_undegraded":"2024-09-22T01:17:18.568759+0000","last_fullsized":"2024-09-22T01:17:18.568759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4305,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.574299+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:18.574299+0000","last_peered":"2024-09-22T01:17:18.574299+0000","last_clean":"2024-09-22T01:17:18.574299+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:18.574299+0000","last_undegraded":"2024-09-22T01:17:18.574299+0000","last_fullsized":"2024-09-22T01:17:18.574299+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19411,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552784+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:17.552784+0000","last_peered":"2024-09-22T01:17:17.552784+0000","last_clean":"2024-09-22T01:17:17.552784+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:17.552784+0000","last_undegraded":"2024-09-22T01:17:17.552784+0000","last_fullsized":"2024-09-22T01:17:17.552784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4580,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552943+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:17.552943+0000","last_peered":"2024-09-22T01:17:17.552943+0000","last_clean":"2024-09-22T01:17:17.552943+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:17.552943+0000","last_undegraded":"2024-09-22T01:17:17.552943+0000","last_fullsized":"2024-09-22T01:17:17.552943+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4962,"reported_epoch":194,"state":"peering","last_fresh":"2024-09-22T01:17:18.077855+0000","last_change":"2024-09-22T01:17:18.587798+0000","last_active":"2024-09-22T01:17:18.587798+0000","last_peered":"2024-09-22T01:17:18.077855+0000","last_clean":"2024-09-22T01:17:11.824095+0000","last_became_active":"2024-09-22T01:17:15.893174+0000","last_became_peered":"2024-09-22T01:17:15.893174+0000","last_unstale":"2024-09-22T01:17:18.077855+0000","last_undegraded":"2024-09-22T01:17:18.077855+0000","last_fullsized":"2024-09-22T01:17:18.077855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6316,"reported_epoch":192,"state":"peering","last_fresh":"2024-09-22T01:17:16.554559+0000","last_change":"2024-09-22T01:17:17.549707+0000","last_active":"2024-09-22T01:17:17.549707+0000","last_peered":"2024-09-22T01:17:16.554559+0000","last_clean":"2024-09-22T01:17:12.838920+0000","last_became_active":"2024-09-22T01:17:15.877839+0000","last_became_peered":"2024-09-22T01:17:15.877839+0000","last_unstale":"2024-09-22T01:17:16.554559+0000","last_undegraded":"2024-09-22T01:17:16.554559+0000","last_fullsized":"2024-09-22T01:17:15.553269+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3047,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553236+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:17.553236+0000","last_peered":"2024-09-22T01:17:17.553236+0000","last_clean":"2024-09-22T01:17:17.553236+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:17.553236+0000","last_undegraded":"2024-09-22T01:17:17.553236+0000","last_fullsized":"2024-09-22T01:17:17.553236+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560061+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:18.560061+0000","last_peered":"2024-09-22T01:17:18.560061+0000","last_clean":"2024-09-22T01:17:18.560061+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:18.560061+0000","last_undegraded":"2024-09-22T01:17:18.560061+0000","last_fullsized":"2024-09-22T01:17:18.560061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4099,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549705+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:17.549705+0000","last_peered":"2024-09-22T01:17:17.549705+0000","last_clean":"2024-09-22T01:17:17.549705+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:17.549705+0000","last_undegraded":"2024-09-22T01:17:17.549705+0000","last_fullsized":"2024-09-22T01:17:17.549705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3853,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568751+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:18.568751+0000","last_peered":"2024-09-22T01:17:18.568751+0000","last_clean":"2024-09-22T01:17:18.568751+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:18.568751+0000","last_undegraded":"2024-09-22T01:17:18.568751+0000","last_fullsized":"2024-09-22T01:17:18.568751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5316,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552682+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:17.552682+0000","last_peered":"2024-09-22T01:17:17.552682+0000","last_clean":"2024-09-22T01:17:17.552682+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:17.552682+0000","last_undegraded":"2024-09-22T01:17:17.552682+0000","last_fullsized":"2024-09-22T01:17:17.552682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5316,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558626+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:18.558626+0000","last_peered":"2024-09-22T01:17:18.558626+0000","last_clean":"2024-09-22T01:17:18.558626+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:18.558626+0000","last_undegraded":"2024-09-22T01:17:18.558626+0000","last_fullsized":"2024-09-22T01:17:18.558626+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6042,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553089+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:17.553089+0000","last_peered":"2024-09-22T01:17:17.553089+0000","last_clean":"2024-09-22T01:17:17.553089+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:17.553089+0000","last_undegraded":"2024-09-22T01:17:17.553089+0000","last_fullsized":"2024-09-22T01:17:17.553089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2314240,"data_stored":1112736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52088,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319752,"statfs":{"total":96636764160,"available":96583426048,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1118769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56248,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315592,"statfs":{"total":96636764160,"available":96579166208,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2310144,"data_stored":1112717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:19.640 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T01:17:19.990 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:19.990 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":225,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":195,"num_osds":8,"num_up_osds":8,"osd_up_since":1726967627,"num_in_osds":8,"osd_in_since":1726967677,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":36},{"state_name":"remapped+peering","count":5}],"num_pgs":41,"num_pools":3,"num_objects":537,"data_bytes":590406,"bytes_used":424329216,"bytes_avail":772669784064,"bytes_total":773094113280,"inactive_pgs_ratio":0.12195122241973877},"fsmap":{"epoch":1,"btime":"2024-09-22T01:13:34:751393+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T01:14:49.536938+0000","services":{}},"progress_events":{}} 2024-09-22T01:17:20.008 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-22T01:17:20.008 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:20.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:20.307 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:20.324 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-22T01:17:19.583495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402056,"kb_used_data":19296,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572664,"statfs":{"total":773094113280,"available":772682407936,"internally_reserved":0,"allocated":19759104,"data_stored":10118746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040487"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3319,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562815+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:18.562815+0000","last_peered":"2024-09-22T01:17:18.562815+0000","last_clean":"2024-09-22T01:17:18.562815+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:18.562815+0000","last_undegraded":"2024-09-22T01:17:18.562815+0000","last_fullsized":"2024-09-22T01:17:18.562815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12828,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562766+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:18.562766+0000","last_peered":"2024-09-22T01:17:18.562766+0000","last_clean":"2024-09-22T01:17:18.562766+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:18.562766+0000","last_undegraded":"2024-09-22T01:17:18.562766+0000","last_fullsized":"2024-09-22T01:17:18.562766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20548,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553190+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:17.553190+0000","last_peered":"2024-09-22T01:17:17.553190+0000","last_clean":"2024-09-22T01:17:17.553190+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:17.553190+0000","last_undegraded":"2024-09-22T01:17:17.553190+0000","last_fullsized":"2024-09-22T01:17:17.553190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13033,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.569031+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:18.569031+0000","last_peered":"2024-09-22T01:17:18.569031+0000","last_clean":"2024-09-22T01:17:18.569031+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:18.569031+0000","last_undegraded":"2024-09-22T01:17:18.569031+0000","last_fullsized":"2024-09-22T01:17:18.569031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5867,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564341+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:18.564341+0000","last_peered":"2024-09-22T01:17:18.564341+0000","last_clean":"2024-09-22T01:17:18.564341+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:18.564341+0000","last_undegraded":"2024-09-22T01:17:18.564341+0000","last_fullsized":"2024-09-22T01:17:18.564341+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17419,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568835+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:18.568835+0000","last_peered":"2024-09-22T01:17:18.568835+0000","last_clean":"2024-09-22T01:17:18.568835+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:18.568835+0000","last_undegraded":"2024-09-22T01:17:18.568835+0000","last_fullsized":"2024-09-22T01:17:18.568835+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6610,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563898+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:18.563898+0000","last_peered":"2024-09-22T01:17:18.563898+0000","last_clean":"2024-09-22T01:17:18.563898+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:18.563898+0000","last_undegraded":"2024-09-22T01:17:18.563898+0000","last_fullsized":"2024-09-22T01:17:18.563898+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564236+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:18.564236+0000","last_peered":"2024-09-22T01:17:18.564236+0000","last_clean":"2024-09-22T01:17:18.564236+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:18.564236+0000","last_undegraded":"2024-09-22T01:17:18.564236+0000","last_fullsized":"2024-09-22T01:17:18.564236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5800,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562939+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:18.562939+0000","last_peered":"2024-09-22T01:17:18.562939+0000","last_clean":"2024-09-22T01:17:18.562939+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:18.562939+0000","last_undegraded":"2024-09-22T01:17:18.562939+0000","last_fullsized":"2024-09-22T01:17:18.562939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13507,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.559668+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:18.559668+0000","last_peered":"2024-09-22T01:17:18.559668+0000","last_clean":"2024-09-22T01:17:18.559668+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:18.559668+0000","last_undegraded":"2024-09-22T01:17:18.559668+0000","last_fullsized":"2024-09-22T01:17:18.559668+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4122,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549410+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:17.549410+0000","last_peered":"2024-09-22T01:17:17.549410+0000","last_clean":"2024-09-22T01:17:17.549410+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:17.549410+0000","last_undegraded":"2024-09-22T01:17:17.549410+0000","last_fullsized":"2024-09-22T01:17:17.549410+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10657,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553351+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:17.553351+0000","last_peered":"2024-09-22T01:17:17.553351+0000","last_clean":"2024-09-22T01:17:17.553351+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:17.553351+0000","last_undegraded":"2024-09-22T01:17:17.553351+0000","last_fullsized":"2024-09-22T01:17:17.553351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552665+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:17.552665+0000","last_peered":"2024-09-22T01:17:17.552665+0000","last_clean":"2024-09-22T01:17:17.552665+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:17.552665+0000","last_undegraded":"2024-09-22T01:17:17.552665+0000","last_fullsized":"2024-09-22T01:17:17.552665+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4951,"reported_epoch":193,"state":"peering","last_fresh":"2024-09-22T01:17:18.484091+0000","last_change":"2024-09-22T01:17:18.565060+0000","last_active":"2024-09-22T01:17:18.565060+0000","last_peered":"2024-09-22T01:17:18.484091+0000","last_clean":"2024-09-22T01:17:12.827374+0000","last_became_active":"2024-09-22T01:17:15.893050+0000","last_became_peered":"2024-09-22T01:17:15.893050+0000","last_unstale":"2024-09-22T01:17:18.484091+0000","last_undegraded":"2024-09-22T01:17:18.484091+0000","last_fullsized":"2024-09-22T01:17:15.546221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564276+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:18.564276+0000","last_peered":"2024-09-22T01:17:18.564276+0000","last_clean":"2024-09-22T01:17:18.564276+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:18.564276+0000","last_undegraded":"2024-09-22T01:17:18.564276+0000","last_fullsized":"2024-09-22T01:17:18.564276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564638+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:18.564638+0000","last_peered":"2024-09-22T01:17:18.564638+0000","last_clean":"2024-09-22T01:17:18.564638+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:18.564638+0000","last_undegraded":"2024-09-22T01:17:18.564638+0000","last_fullsized":"2024-09-22T01:17:18.564638+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553279+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:17.553279+0000","last_peered":"2024-09-22T01:17:17.553279+0000","last_clean":"2024-09-22T01:17:17.553279+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:17.553279+0000","last_undegraded":"2024-09-22T01:17:17.553279+0000","last_fullsized":"2024-09-22T01:17:17.553279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3692,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558357+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:18.558357+0000","last_peered":"2024-09-22T01:17:18.558357+0000","last_clean":"2024-09-22T01:17:18.558357+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:18.558357+0000","last_undegraded":"2024-09-22T01:17:18.558357+0000","last_fullsized":"2024-09-22T01:17:18.558357+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":358,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562708+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:18.562708+0000","last_peered":"2024-09-22T01:17:18.562708+0000","last_clean":"2024-09-22T01:17:18.562708+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:18.562708+0000","last_undegraded":"2024-09-22T01:17:18.562708+0000","last_fullsized":"2024-09-22T01:17:18.562708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8759,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560762+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:18.560762+0000","last_peered":"2024-09-22T01:17:18.560762+0000","last_clean":"2024-09-22T01:17:18.560762+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:18.560762+0000","last_undegraded":"2024-09-22T01:17:18.560762+0000","last_fullsized":"2024-09-22T01:17:18.560762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6978,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.550623+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:17.550623+0000","last_peered":"2024-09-22T01:17:17.550623+0000","last_clean":"2024-09-22T01:17:17.550623+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:17.550623+0000","last_undegraded":"2024-09-22T01:17:17.550623+0000","last_fullsized":"2024-09-22T01:17:17.550623+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553031+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:17.553031+0000","last_peered":"2024-09-22T01:17:17.553031+0000","last_clean":"2024-09-22T01:17:17.553031+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:17.553031+0000","last_undegraded":"2024-09-22T01:17:17.553031+0000","last_fullsized":"2024-09-22T01:17:17.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4604,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.573188+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:18.573188+0000","last_peered":"2024-09-22T01:17:18.573188+0000","last_clean":"2024-09-22T01:17:18.573188+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:18.573188+0000","last_undegraded":"2024-09-22T01:17:18.573188+0000","last_fullsized":"2024-09-22T01:17:18.573188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558136+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:18.558136+0000","last_peered":"2024-09-22T01:17:18.558136+0000","last_clean":"2024-09-22T01:17:18.558136+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:18.558136+0000","last_undegraded":"2024-09-22T01:17:18.558136+0000","last_fullsized":"2024-09-22T01:17:18.558136+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563777+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:18.563777+0000","last_peered":"2024-09-22T01:17:18.563777+0000","last_clean":"2024-09-22T01:17:18.563777+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:18.563777+0000","last_undegraded":"2024-09-22T01:17:18.563777+0000","last_fullsized":"2024-09-22T01:17:18.563777+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553473+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:17.553473+0000","last_peered":"2024-09-22T01:17:17.553473+0000","last_clean":"2024-09-22T01:17:17.553473+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:17.553473+0000","last_undegraded":"2024-09-22T01:17:17.553473+0000","last_fullsized":"2024-09-22T01:17:17.553473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6576,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564025+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:18.564025+0000","last_peered":"2024-09-22T01:17:18.564025+0000","last_clean":"2024-09-22T01:17:18.564025+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:18.564025+0000","last_undegraded":"2024-09-22T01:17:18.564025+0000","last_fullsized":"2024-09-22T01:17:18.564025+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":359,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553135+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:17.553135+0000","last_peered":"2024-09-22T01:17:17.553135+0000","last_clean":"2024-09-22T01:17:17.553135+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:17.553135+0000","last_undegraded":"2024-09-22T01:17:17.553135+0000","last_fullsized":"2024-09-22T01:17:17.553135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":511,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568759+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:18.568759+0000","last_peered":"2024-09-22T01:17:18.568759+0000","last_clean":"2024-09-22T01:17:18.568759+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:18.568759+0000","last_undegraded":"2024-09-22T01:17:18.568759+0000","last_fullsized":"2024-09-22T01:17:18.568759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4305,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.574299+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:18.574299+0000","last_peered":"2024-09-22T01:17:18.574299+0000","last_clean":"2024-09-22T01:17:18.574299+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:18.574299+0000","last_undegraded":"2024-09-22T01:17:18.574299+0000","last_fullsized":"2024-09-22T01:17:18.574299+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19411,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552784+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:17.552784+0000","last_peered":"2024-09-22T01:17:17.552784+0000","last_clean":"2024-09-22T01:17:17.552784+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:17.552784+0000","last_undegraded":"2024-09-22T01:17:17.552784+0000","last_fullsized":"2024-09-22T01:17:17.552784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4580,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552943+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:17.552943+0000","last_peered":"2024-09-22T01:17:17.552943+0000","last_clean":"2024-09-22T01:17:17.552943+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:17.552943+0000","last_undegraded":"2024-09-22T01:17:17.552943+0000","last_fullsized":"2024-09-22T01:17:17.552943+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4962,"reported_epoch":194,"state":"peering","last_fresh":"2024-09-22T01:17:18.077855+0000","last_change":"2024-09-22T01:17:18.587798+0000","last_active":"2024-09-22T01:17:18.587798+0000","last_peered":"2024-09-22T01:17:18.077855+0000","last_clean":"2024-09-22T01:17:11.824095+0000","last_became_active":"2024-09-22T01:17:15.893174+0000","last_became_peered":"2024-09-22T01:17:15.893174+0000","last_unstale":"2024-09-22T01:17:18.077855+0000","last_undegraded":"2024-09-22T01:17:18.077855+0000","last_fullsized":"2024-09-22T01:17:18.077855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6316,"reported_epoch":192,"state":"peering","last_fresh":"2024-09-22T01:17:16.554559+0000","last_change":"2024-09-22T01:17:17.549707+0000","last_active":"2024-09-22T01:17:17.549707+0000","last_peered":"2024-09-22T01:17:16.554559+0000","last_clean":"2024-09-22T01:17:12.838920+0000","last_became_active":"2024-09-22T01:17:15.877839+0000","last_became_peered":"2024-09-22T01:17:15.877839+0000","last_unstale":"2024-09-22T01:17:16.554559+0000","last_undegraded":"2024-09-22T01:17:16.554559+0000","last_fullsized":"2024-09-22T01:17:15.553269+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3047,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553236+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:17.553236+0000","last_peered":"2024-09-22T01:17:17.553236+0000","last_clean":"2024-09-22T01:17:17.553236+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:17.553236+0000","last_undegraded":"2024-09-22T01:17:17.553236+0000","last_fullsized":"2024-09-22T01:17:17.553236+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560061+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:18.560061+0000","last_peered":"2024-09-22T01:17:18.560061+0000","last_clean":"2024-09-22T01:17:18.560061+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:18.560061+0000","last_undegraded":"2024-09-22T01:17:18.560061+0000","last_fullsized":"2024-09-22T01:17:18.560061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4099,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.549705+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:17.549705+0000","last_peered":"2024-09-22T01:17:17.549705+0000","last_clean":"2024-09-22T01:17:17.549705+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:17.549705+0000","last_undegraded":"2024-09-22T01:17:17.549705+0000","last_fullsized":"2024-09-22T01:17:17.549705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3853,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568751+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:18.568751+0000","last_peered":"2024-09-22T01:17:18.568751+0000","last_clean":"2024-09-22T01:17:18.568751+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:18.568751+0000","last_undegraded":"2024-09-22T01:17:18.568751+0000","last_fullsized":"2024-09-22T01:17:18.568751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5316,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552682+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:17.552682+0000","last_peered":"2024-09-22T01:17:17.552682+0000","last_clean":"2024-09-22T01:17:17.552682+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:17.552682+0000","last_undegraded":"2024-09-22T01:17:17.552682+0000","last_fullsized":"2024-09-22T01:17:17.552682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5316,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558626+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:18.558626+0000","last_peered":"2024-09-22T01:17:18.558626+0000","last_clean":"2024-09-22T01:17:18.558626+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:18.558626+0000","last_undegraded":"2024-09-22T01:17:18.558626+0000","last_fullsized":"2024-09-22T01:17:18.558626+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6042,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553089+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:17.553089+0000","last_peered":"2024-09-22T01:17:17.553089+0000","last_clean":"2024-09-22T01:17:17.553089+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:17.553089+0000","last_undegraded":"2024-09-22T01:17:17.553089+0000","last_fullsized":"2024-09-22T01:17:17.553089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2314240,"data_stored":1112736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52088,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319752,"statfs":{"total":96636764160,"available":96583426048,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1118769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56248,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315592,"statfs":{"total":96636764160,"available":96579166208,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59812,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312028,"statfs":{"total":96636764160,"available":96575516672,"internally_reserved":0,"allocated":2310144,"data_stored":1112717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:23.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:23.640 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:23.640 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:23.659 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-22T01:17:23.584777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402096,"kb_used_data":19332,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572624,"statfs":{"total":773094113280,"available":772682366976,"internally_reserved":0,"allocated":19795968,"data_stored":10136645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038225"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3319,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562815+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:18.562815+0000","last_peered":"2024-09-22T01:17:18.562815+0000","last_clean":"2024-09-22T01:17:18.562815+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:18.562815+0000","last_undegraded":"2024-09-22T01:17:18.562815+0000","last_fullsized":"2024-09-22T01:17:18.562815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12828,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562766+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:18.562766+0000","last_peered":"2024-09-22T01:17:18.562766+0000","last_clean":"2024-09-22T01:17:18.562766+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:18.562766+0000","last_undegraded":"2024-09-22T01:17:18.562766+0000","last_fullsized":"2024-09-22T01:17:18.562766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20548,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553190+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:17.553190+0000","last_peered":"2024-09-22T01:17:17.553190+0000","last_clean":"2024-09-22T01:17:17.553190+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:17.553190+0000","last_undegraded":"2024-09-22T01:17:17.553190+0000","last_fullsized":"2024-09-22T01:17:17.553190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13033,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.569031+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:18.569031+0000","last_peered":"2024-09-22T01:17:18.569031+0000","last_clean":"2024-09-22T01:17:18.569031+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:18.569031+0000","last_undegraded":"2024-09-22T01:17:18.569031+0000","last_fullsized":"2024-09-22T01:17:18.569031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5867,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564341+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:18.564341+0000","last_peered":"2024-09-22T01:17:18.564341+0000","last_clean":"2024-09-22T01:17:18.564341+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:18.564341+0000","last_undegraded":"2024-09-22T01:17:18.564341+0000","last_fullsized":"2024-09-22T01:17:18.564341+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17419,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568835+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:18.568835+0000","last_peered":"2024-09-22T01:17:18.568835+0000","last_clean":"2024-09-22T01:17:18.568835+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:18.568835+0000","last_undegraded":"2024-09-22T01:17:18.568835+0000","last_fullsized":"2024-09-22T01:17:18.568835+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6610,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563898+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:18.563898+0000","last_peered":"2024-09-22T01:17:18.563898+0000","last_clean":"2024-09-22T01:17:18.563898+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:18.563898+0000","last_undegraded":"2024-09-22T01:17:18.563898+0000","last_fullsized":"2024-09-22T01:17:18.563898+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564236+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:18.564236+0000","last_peered":"2024-09-22T01:17:18.564236+0000","last_clean":"2024-09-22T01:17:18.564236+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:18.564236+0000","last_undegraded":"2024-09-22T01:17:18.564236+0000","last_fullsized":"2024-09-22T01:17:18.564236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5800,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562939+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:18.562939+0000","last_peered":"2024-09-22T01:17:18.562939+0000","last_clean":"2024-09-22T01:17:18.562939+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:18.562939+0000","last_undegraded":"2024-09-22T01:17:18.562939+0000","last_fullsized":"2024-09-22T01:17:18.562939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13507,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.559668+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:18.559668+0000","last_peered":"2024-09-22T01:17:18.559668+0000","last_clean":"2024-09-22T01:17:18.559668+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:18.559668+0000","last_undegraded":"2024-09-22T01:17:18.559668+0000","last_fullsized":"2024-09-22T01:17:18.559668+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10657,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553351+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:17.553351+0000","last_peered":"2024-09-22T01:17:17.553351+0000","last_clean":"2024-09-22T01:17:17.553351+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:17.553351+0000","last_undegraded":"2024-09-22T01:17:17.553351+0000","last_fullsized":"2024-09-22T01:17:17.553351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552665+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:17.552665+0000","last_peered":"2024-09-22T01:17:17.552665+0000","last_clean":"2024-09-22T01:17:17.552665+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:17.552665+0000","last_undegraded":"2024-09-22T01:17:17.552665+0000","last_fullsized":"2024-09-22T01:17:17.552665+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4951,"reported_epoch":193,"state":"peering","last_fresh":"2024-09-22T01:17:18.484091+0000","last_change":"2024-09-22T01:17:18.565060+0000","last_active":"2024-09-22T01:17:18.565060+0000","last_peered":"2024-09-22T01:17:18.484091+0000","last_clean":"2024-09-22T01:17:12.827374+0000","last_became_active":"2024-09-22T01:17:15.893050+0000","last_became_peered":"2024-09-22T01:17:15.893050+0000","last_unstale":"2024-09-22T01:17:18.484091+0000","last_undegraded":"2024-09-22T01:17:18.484091+0000","last_fullsized":"2024-09-22T01:17:15.546221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564276+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:18.564276+0000","last_peered":"2024-09-22T01:17:18.564276+0000","last_clean":"2024-09-22T01:17:18.564276+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:18.564276+0000","last_undegraded":"2024-09-22T01:17:18.564276+0000","last_fullsized":"2024-09-22T01:17:18.564276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564638+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:18.564638+0000","last_peered":"2024-09-22T01:17:18.564638+0000","last_clean":"2024-09-22T01:17:18.564638+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:18.564638+0000","last_undegraded":"2024-09-22T01:17:18.564638+0000","last_fullsized":"2024-09-22T01:17:18.564638+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553279+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:17.553279+0000","last_peered":"2024-09-22T01:17:17.553279+0000","last_clean":"2024-09-22T01:17:17.553279+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:17.553279+0000","last_undegraded":"2024-09-22T01:17:17.553279+0000","last_fullsized":"2024-09-22T01:17:17.553279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3692,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558357+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:18.558357+0000","last_peered":"2024-09-22T01:17:18.558357+0000","last_clean":"2024-09-22T01:17:18.558357+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:18.558357+0000","last_undegraded":"2024-09-22T01:17:18.558357+0000","last_fullsized":"2024-09-22T01:17:18.558357+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":358,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562708+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:18.562708+0000","last_peered":"2024-09-22T01:17:18.562708+0000","last_clean":"2024-09-22T01:17:18.562708+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:18.562708+0000","last_undegraded":"2024-09-22T01:17:18.562708+0000","last_fullsized":"2024-09-22T01:17:18.562708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8759,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560762+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:18.560762+0000","last_peered":"2024-09-22T01:17:18.560762+0000","last_clean":"2024-09-22T01:17:18.560762+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:18.560762+0000","last_undegraded":"2024-09-22T01:17:18.560762+0000","last_fullsized":"2024-09-22T01:17:18.560762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553031+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:17.553031+0000","last_peered":"2024-09-22T01:17:17.553031+0000","last_clean":"2024-09-22T01:17:17.553031+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:17.553031+0000","last_undegraded":"2024-09-22T01:17:17.553031+0000","last_fullsized":"2024-09-22T01:17:17.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4604,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.573188+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:18.573188+0000","last_peered":"2024-09-22T01:17:18.573188+0000","last_clean":"2024-09-22T01:17:18.573188+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:18.573188+0000","last_undegraded":"2024-09-22T01:17:18.573188+0000","last_fullsized":"2024-09-22T01:17:18.573188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558136+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:18.558136+0000","last_peered":"2024-09-22T01:17:18.558136+0000","last_clean":"2024-09-22T01:17:18.558136+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:18.558136+0000","last_undegraded":"2024-09-22T01:17:18.558136+0000","last_fullsized":"2024-09-22T01:17:18.558136+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563777+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:18.563777+0000","last_peered":"2024-09-22T01:17:18.563777+0000","last_clean":"2024-09-22T01:17:18.563777+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:18.563777+0000","last_undegraded":"2024-09-22T01:17:18.563777+0000","last_fullsized":"2024-09-22T01:17:18.563777+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553473+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:17.553473+0000","last_peered":"2024-09-22T01:17:17.553473+0000","last_clean":"2024-09-22T01:17:17.553473+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:17.553473+0000","last_undegraded":"2024-09-22T01:17:17.553473+0000","last_fullsized":"2024-09-22T01:17:17.553473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6576,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564025+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:18.564025+0000","last_peered":"2024-09-22T01:17:18.564025+0000","last_clean":"2024-09-22T01:17:18.564025+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:18.564025+0000","last_undegraded":"2024-09-22T01:17:18.564025+0000","last_fullsized":"2024-09-22T01:17:18.564025+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":359,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553135+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:17.553135+0000","last_peered":"2024-09-22T01:17:17.553135+0000","last_clean":"2024-09-22T01:17:17.553135+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:17.553135+0000","last_undegraded":"2024-09-22T01:17:17.553135+0000","last_fullsized":"2024-09-22T01:17:17.553135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":511,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568759+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:18.568759+0000","last_peered":"2024-09-22T01:17:18.568759+0000","last_clean":"2024-09-22T01:17:18.568759+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:18.568759+0000","last_undegraded":"2024-09-22T01:17:18.568759+0000","last_fullsized":"2024-09-22T01:17:18.568759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4305,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.574299+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:18.574299+0000","last_peered":"2024-09-22T01:17:18.574299+0000","last_clean":"2024-09-22T01:17:18.574299+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:18.574299+0000","last_undegraded":"2024-09-22T01:17:18.574299+0000","last_fullsized":"2024-09-22T01:17:18.574299+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19411,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552784+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:17.552784+0000","last_peered":"2024-09-22T01:17:17.552784+0000","last_clean":"2024-09-22T01:17:17.552784+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:17.552784+0000","last_undegraded":"2024-09-22T01:17:17.552784+0000","last_fullsized":"2024-09-22T01:17:17.552784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4580,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552943+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:17.552943+0000","last_peered":"2024-09-22T01:17:17.552943+0000","last_clean":"2024-09-22T01:17:17.552943+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:17.552943+0000","last_undegraded":"2024-09-22T01:17:17.552943+0000","last_fullsized":"2024-09-22T01:17:17.552943+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4962,"reported_epoch":194,"state":"peering","last_fresh":"2024-09-22T01:17:18.077855+0000","last_change":"2024-09-22T01:17:18.587798+0000","last_active":"2024-09-22T01:17:18.587798+0000","last_peered":"2024-09-22T01:17:18.077855+0000","last_clean":"2024-09-22T01:17:11.824095+0000","last_became_active":"2024-09-22T01:17:15.893174+0000","last_became_peered":"2024-09-22T01:17:15.893174+0000","last_unstale":"2024-09-22T01:17:18.077855+0000","last_undegraded":"2024-09-22T01:17:18.077855+0000","last_fullsized":"2024-09-22T01:17:18.077855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3047,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553236+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:17.553236+0000","last_peered":"2024-09-22T01:17:17.553236+0000","last_clean":"2024-09-22T01:17:17.553236+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:17.553236+0000","last_undegraded":"2024-09-22T01:17:17.553236+0000","last_fullsized":"2024-09-22T01:17:17.553236+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560061+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:18.560061+0000","last_peered":"2024-09-22T01:17:18.560061+0000","last_clean":"2024-09-22T01:17:18.560061+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:18.560061+0000","last_undegraded":"2024-09-22T01:17:18.560061+0000","last_fullsized":"2024-09-22T01:17:18.560061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3853,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568751+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:18.568751+0000","last_peered":"2024-09-22T01:17:18.568751+0000","last_clean":"2024-09-22T01:17:18.568751+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:18.568751+0000","last_undegraded":"2024-09-22T01:17:18.568751+0000","last_fullsized":"2024-09-22T01:17:18.568751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5316,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552682+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:17.552682+0000","last_peered":"2024-09-22T01:17:17.552682+0000","last_clean":"2024-09-22T01:17:17.552682+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:17.552682+0000","last_undegraded":"2024-09-22T01:17:17.552682+0000","last_fullsized":"2024-09-22T01:17:17.552682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5316,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558626+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:18.558626+0000","last_peered":"2024-09-22T01:17:18.558626+0000","last_clean":"2024-09-22T01:17:18.558626+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:18.558626+0000","last_undegraded":"2024-09-22T01:17:18.558626+0000","last_fullsized":"2024-09-22T01:17:18.558626+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6042,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553089+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:17.553089+0000","last_peered":"2024-09-22T01:17:17.553089+0000","last_clean":"2024-09-22T01:17:17.553089+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:17.553089+0000","last_undegraded":"2024-09-22T01:17:17.553089+0000","last_fullsized":"2024-09-22T01:17:17.553089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2314240,"data_stored":1112736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52088,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319752,"statfs":{"total":96636764160,"available":96583426048,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1118769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56248,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315592,"statfs":{"total":96636764160,"available":96579166208,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:23.661 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-22T01:17:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:23.994 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":229,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":195,"num_osds":8,"num_up_osds":8,"osd_up_since":1726967627,"num_in_osds":8,"osd_in_since":1726967677,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"peering","count":2}],"num_pgs":41,"num_pools":3,"num_objects":537,"data_bytes":590406,"bytes_used":411746304,"bytes_avail":772682366976,"bytes_total":773094113280,"inactive_pgs_ratio":0.048780485987663269,"recovering_objects_per_sec":116,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":702,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-22T01:13:34:751393+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-22T01:14:49.536938+0000","services":{}},"progress_events":{}} 2024-09-22T01:17:24.011 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-22T01:17:24.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:24.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:24.305 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:24.322 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-22T01:17:23.584777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402096,"kb_used_data":19332,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572624,"statfs":{"total":773094113280,"available":772682366976,"internally_reserved":0,"allocated":19795968,"data_stored":10136645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":702,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.038225"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3319,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562815+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:18.562815+0000","last_peered":"2024-09-22T01:17:18.562815+0000","last_clean":"2024-09-22T01:17:18.562815+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:18.562815+0000","last_undegraded":"2024-09-22T01:17:18.562815+0000","last_fullsized":"2024-09-22T01:17:18.562815+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12828,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562766+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:18.562766+0000","last_peered":"2024-09-22T01:17:18.562766+0000","last_clean":"2024-09-22T01:17:18.562766+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:18.562766+0000","last_undegraded":"2024-09-22T01:17:18.562766+0000","last_fullsized":"2024-09-22T01:17:18.562766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20548,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553190+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:17.553190+0000","last_peered":"2024-09-22T01:17:17.553190+0000","last_clean":"2024-09-22T01:17:17.553190+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:17.553190+0000","last_undegraded":"2024-09-22T01:17:17.553190+0000","last_fullsized":"2024-09-22T01:17:17.553190+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13033,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.569031+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:18.569031+0000","last_peered":"2024-09-22T01:17:18.569031+0000","last_clean":"2024-09-22T01:17:18.569031+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:18.569031+0000","last_undegraded":"2024-09-22T01:17:18.569031+0000","last_fullsized":"2024-09-22T01:17:18.569031+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5867,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564341+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:18.564341+0000","last_peered":"2024-09-22T01:17:18.564341+0000","last_clean":"2024-09-22T01:17:18.564341+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:18.564341+0000","last_undegraded":"2024-09-22T01:17:18.564341+0000","last_fullsized":"2024-09-22T01:17:18.564341+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17419,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568835+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:18.568835+0000","last_peered":"2024-09-22T01:17:18.568835+0000","last_clean":"2024-09-22T01:17:18.568835+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:18.568835+0000","last_undegraded":"2024-09-22T01:17:18.568835+0000","last_fullsized":"2024-09-22T01:17:18.568835+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6610,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563898+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:18.563898+0000","last_peered":"2024-09-22T01:17:18.563898+0000","last_clean":"2024-09-22T01:17:18.563898+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:18.563898+0000","last_undegraded":"2024-09-22T01:17:18.563898+0000","last_fullsized":"2024-09-22T01:17:18.563898+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6644,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564236+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:18.564236+0000","last_peered":"2024-09-22T01:17:18.564236+0000","last_clean":"2024-09-22T01:17:18.564236+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:18.564236+0000","last_undegraded":"2024-09-22T01:17:18.564236+0000","last_fullsized":"2024-09-22T01:17:18.564236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5800,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562939+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:18.562939+0000","last_peered":"2024-09-22T01:17:18.562939+0000","last_clean":"2024-09-22T01:17:18.562939+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:18.562939+0000","last_undegraded":"2024-09-22T01:17:18.562939+0000","last_fullsized":"2024-09-22T01:17:18.562939+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13507,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.559668+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:18.559668+0000","last_peered":"2024-09-22T01:17:18.559668+0000","last_clean":"2024-09-22T01:17:18.559668+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:18.559668+0000","last_undegraded":"2024-09-22T01:17:18.559668+0000","last_fullsized":"2024-09-22T01:17:18.559668+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10657,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553351+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:17.553351+0000","last_peered":"2024-09-22T01:17:17.553351+0000","last_clean":"2024-09-22T01:17:17.553351+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:17.553351+0000","last_undegraded":"2024-09-22T01:17:17.553351+0000","last_fullsized":"2024-09-22T01:17:17.553351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552665+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:17.552665+0000","last_peered":"2024-09-22T01:17:17.552665+0000","last_clean":"2024-09-22T01:17:17.552665+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:17.552665+0000","last_undegraded":"2024-09-22T01:17:17.552665+0000","last_fullsized":"2024-09-22T01:17:17.552665+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4951,"reported_epoch":193,"state":"peering","last_fresh":"2024-09-22T01:17:18.484091+0000","last_change":"2024-09-22T01:17:18.565060+0000","last_active":"2024-09-22T01:17:18.565060+0000","last_peered":"2024-09-22T01:17:18.484091+0000","last_clean":"2024-09-22T01:17:12.827374+0000","last_became_active":"2024-09-22T01:17:15.893050+0000","last_became_peered":"2024-09-22T01:17:15.893050+0000","last_unstale":"2024-09-22T01:17:18.484091+0000","last_undegraded":"2024-09-22T01:17:18.484091+0000","last_fullsized":"2024-09-22T01:17:15.546221+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":189,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564276+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:18.564276+0000","last_peered":"2024-09-22T01:17:18.564276+0000","last_clean":"2024-09-22T01:17:18.564276+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:18.564276+0000","last_undegraded":"2024-09-22T01:17:18.564276+0000","last_fullsized":"2024-09-22T01:17:18.564276+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4321,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564638+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:18.564638+0000","last_peered":"2024-09-22T01:17:18.564638+0000","last_clean":"2024-09-22T01:17:18.564638+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:18.564638+0000","last_undegraded":"2024-09-22T01:17:18.564638+0000","last_fullsized":"2024-09-22T01:17:18.564638+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553279+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:17.553279+0000","last_peered":"2024-09-22T01:17:17.553279+0000","last_clean":"2024-09-22T01:17:17.553279+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:17.553279+0000","last_undegraded":"2024-09-22T01:17:17.553279+0000","last_fullsized":"2024-09-22T01:17:17.553279+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3692,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558357+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:18.558357+0000","last_peered":"2024-09-22T01:17:18.558357+0000","last_clean":"2024-09-22T01:17:18.558357+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:18.558357+0000","last_undegraded":"2024-09-22T01:17:18.558357+0000","last_fullsized":"2024-09-22T01:17:18.558357+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":358,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.562708+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:18.562708+0000","last_peered":"2024-09-22T01:17:18.562708+0000","last_clean":"2024-09-22T01:17:18.562708+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:18.562708+0000","last_undegraded":"2024-09-22T01:17:18.562708+0000","last_fullsized":"2024-09-22T01:17:18.562708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8759,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560762+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:18.560762+0000","last_peered":"2024-09-22T01:17:18.560762+0000","last_clean":"2024-09-22T01:17:18.560762+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:18.560762+0000","last_undegraded":"2024-09-22T01:17:18.560762+0000","last_fullsized":"2024-09-22T01:17:18.560762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":341,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553031+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:17.553031+0000","last_peered":"2024-09-22T01:17:17.553031+0000","last_clean":"2024-09-22T01:17:17.553031+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:17.553031+0000","last_undegraded":"2024-09-22T01:17:17.553031+0000","last_fullsized":"2024-09-22T01:17:17.553031+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4604,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.573188+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:18.573188+0000","last_peered":"2024-09-22T01:17:18.573188+0000","last_clean":"2024-09-22T01:17:18.573188+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:18.573188+0000","last_undegraded":"2024-09-22T01:17:18.573188+0000","last_fullsized":"2024-09-22T01:17:18.573188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558136+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:18.558136+0000","last_peered":"2024-09-22T01:17:18.558136+0000","last_clean":"2024-09-22T01:17:18.558136+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:18.558136+0000","last_undegraded":"2024-09-22T01:17:18.558136+0000","last_fullsized":"2024-09-22T01:17:18.558136+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7034,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.563777+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:18.563777+0000","last_peered":"2024-09-22T01:17:18.563777+0000","last_clean":"2024-09-22T01:17:18.563777+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:18.563777+0000","last_undegraded":"2024-09-22T01:17:18.563777+0000","last_fullsized":"2024-09-22T01:17:18.563777+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":337,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553473+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:17.553473+0000","last_peered":"2024-09-22T01:17:17.553473+0000","last_clean":"2024-09-22T01:17:17.553473+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:17.553473+0000","last_undegraded":"2024-09-22T01:17:17.553473+0000","last_fullsized":"2024-09-22T01:17:17.553473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6576,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.564025+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:18.564025+0000","last_peered":"2024-09-22T01:17:18.564025+0000","last_clean":"2024-09-22T01:17:18.564025+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:18.564025+0000","last_undegraded":"2024-09-22T01:17:18.564025+0000","last_fullsized":"2024-09-22T01:17:18.564025+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":359,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553135+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:17.553135+0000","last_peered":"2024-09-22T01:17:17.553135+0000","last_clean":"2024-09-22T01:17:17.553135+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:17.553135+0000","last_undegraded":"2024-09-22T01:17:17.553135+0000","last_fullsized":"2024-09-22T01:17:17.553135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":511,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568759+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:18.568759+0000","last_peered":"2024-09-22T01:17:18.568759+0000","last_clean":"2024-09-22T01:17:18.568759+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:18.568759+0000","last_undegraded":"2024-09-22T01:17:18.568759+0000","last_fullsized":"2024-09-22T01:17:18.568759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4305,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.574299+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:18.574299+0000","last_peered":"2024-09-22T01:17:18.574299+0000","last_clean":"2024-09-22T01:17:18.574299+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:18.574299+0000","last_undegraded":"2024-09-22T01:17:18.574299+0000","last_fullsized":"2024-09-22T01:17:18.574299+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19411,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552784+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:17.552784+0000","last_peered":"2024-09-22T01:17:17.552784+0000","last_clean":"2024-09-22T01:17:17.552784+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:17.552784+0000","last_undegraded":"2024-09-22T01:17:17.552784+0000","last_fullsized":"2024-09-22T01:17:17.552784+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4580,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552943+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:17.552943+0000","last_peered":"2024-09-22T01:17:17.552943+0000","last_clean":"2024-09-22T01:17:17.552943+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:17.552943+0000","last_undegraded":"2024-09-22T01:17:17.552943+0000","last_fullsized":"2024-09-22T01:17:17.552943+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4962,"reported_epoch":194,"state":"peering","last_fresh":"2024-09-22T01:17:18.077855+0000","last_change":"2024-09-22T01:17:18.587798+0000","last_active":"2024-09-22T01:17:18.587798+0000","last_peered":"2024-09-22T01:17:18.077855+0000","last_clean":"2024-09-22T01:17:11.824095+0000","last_became_active":"2024-09-22T01:17:15.893174+0000","last_became_peered":"2024-09-22T01:17:15.893174+0000","last_unstale":"2024-09-22T01:17:18.077855+0000","last_undegraded":"2024-09-22T01:17:18.077855+0000","last_fullsized":"2024-09-22T01:17:18.077855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3047,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553236+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:17.553236+0000","last_peered":"2024-09-22T01:17:17.553236+0000","last_clean":"2024-09-22T01:17:17.553236+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:17.553236+0000","last_undegraded":"2024-09-22T01:17:17.553236+0000","last_fullsized":"2024-09-22T01:17:17.553236+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.560061+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:18.560061+0000","last_peered":"2024-09-22T01:17:18.560061+0000","last_clean":"2024-09-22T01:17:18.560061+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:18.560061+0000","last_undegraded":"2024-09-22T01:17:18.560061+0000","last_fullsized":"2024-09-22T01:17:18.560061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3853,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.568751+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:18.568751+0000","last_peered":"2024-09-22T01:17:18.568751+0000","last_clean":"2024-09-22T01:17:18.568751+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:18.568751+0000","last_undegraded":"2024-09-22T01:17:18.568751+0000","last_fullsized":"2024-09-22T01:17:18.568751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5316,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.552682+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:17.552682+0000","last_peered":"2024-09-22T01:17:17.552682+0000","last_clean":"2024-09-22T01:17:17.552682+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:17.552682+0000","last_undegraded":"2024-09-22T01:17:17.552682+0000","last_fullsized":"2024-09-22T01:17:17.552682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5316,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-09-22T01:17:18.558626+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:18.558626+0000","last_peered":"2024-09-22T01:17:18.558626+0000","last_clean":"2024-09-22T01:17:18.558626+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:18.558626+0000","last_undegraded":"2024-09-22T01:17:18.558626+0000","last_fullsized":"2024-09-22T01:17:18.558626+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6042,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-09-22T01:17:17.553089+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:17.553089+0000","last_peered":"2024-09-22T01:17:17.553089+0000","last_clean":"2024-09-22T01:17:17.553089+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:17.553089+0000","last_undegraded":"2024-09-22T01:17:17.553089+0000","last_fullsized":"2024-09-22T01:17:17.553089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29296,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342544,"statfs":{"total":96636764160,"available":96606765056,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51504,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320336,"statfs":{"total":96636764160,"available":96584024064,"internally_reserved":0,"allocated":2314240,"data_stored":1112736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52088,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319752,"statfs":{"total":96636764160,"available":96583426048,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33456,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1118769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509488,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1118750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56248,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315592,"statfs":{"total":96636764160,"available":96579166208,"internally_reserved":0,"allocated":2920448,"data_stored":1709137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542192,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:27.325 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:27.633 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:27.634 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:27.650 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T01:17:27.586238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402300,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572420,"statfs":{"total":773094113280,"available":772682158080,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-11,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.030731"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3321,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169634+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:20.169634+0000","last_peered":"2024-09-22T01:17:20.169634+0000","last_clean":"2024-09-22T01:17:20.169634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:20.169634+0000","last_undegraded":"2024-09-22T01:17:20.169634+0000","last_fullsized":"2024-09-22T01:17:20.169634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12830,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169570+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:20.169570+0000","last_peered":"2024-09-22T01:17:20.169570+0000","last_clean":"2024-09-22T01:17:20.169570+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:20.169570+0000","last_undegraded":"2024-09-22T01:17:20.169570+0000","last_fullsized":"2024-09-22T01:17:20.169570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559885+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:19.559885+0000","last_peered":"2024-09-22T01:17:19.559885+0000","last_clean":"2024-09-22T01:17:19.559885+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:19.559885+0000","last_undegraded":"2024-09-22T01:17:19.559885+0000","last_fullsized":"2024-09-22T01:17:19.559885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168447+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:20.168447+0000","last_peered":"2024-09-22T01:17:20.168447+0000","last_clean":"2024-09-22T01:17:20.168447+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:20.168447+0000","last_undegraded":"2024-09-22T01:17:20.168447+0000","last_fullsized":"2024-09-22T01:17:20.168447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561110+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:19.561110+0000","last_peered":"2024-09-22T01:17:19.561110+0000","last_clean":"2024-09-22T01:17:19.561110+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:19.561110+0000","last_undegraded":"2024-09-22T01:17:19.561110+0000","last_fullsized":"2024-09-22T01:17:19.561110+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17421,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168499+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:20.168499+0000","last_peered":"2024-09-22T01:17:20.168499+0000","last_clean":"2024-09-22T01:17:20.168499+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:20.168499+0000","last_undegraded":"2024-09-22T01:17:20.168499+0000","last_fullsized":"2024-09-22T01:17:20.168499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6612,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560970+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:19.560970+0000","last_peered":"2024-09-22T01:17:19.560970+0000","last_clean":"2024-09-22T01:17:19.560970+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:19.560970+0000","last_undegraded":"2024-09-22T01:17:19.560970+0000","last_fullsized":"2024-09-22T01:17:19.560970+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169725+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:20.169725+0000","last_peered":"2024-09-22T01:17:20.169725+0000","last_clean":"2024-09-22T01:17:20.169725+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:20.169725+0000","last_undegraded":"2024-09-22T01:17:20.169725+0000","last_fullsized":"2024-09-22T01:17:20.169725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5802,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169776+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:20.169776+0000","last_peered":"2024-09-22T01:17:20.169776+0000","last_clean":"2024-09-22T01:17:20.169776+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:20.169776+0000","last_undegraded":"2024-09-22T01:17:20.169776+0000","last_fullsized":"2024-09-22T01:17:20.169776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13509,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783820+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:19.783820+0000","last_peered":"2024-09-22T01:17:19.783820+0000","last_clean":"2024-09-22T01:17:19.783820+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:19.783820+0000","last_undegraded":"2024-09-22T01:17:19.783820+0000","last_fullsized":"2024-09-22T01:17:19.783820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10661,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559715+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:19.559715+0000","last_peered":"2024-09-22T01:17:19.559715+0000","last_clean":"2024-09-22T01:17:19.559715+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:19.559715+0000","last_undegraded":"2024-09-22T01:17:19.559715+0000","last_fullsized":"2024-09-22T01:17:19.559715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560176+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:19.560176+0000","last_peered":"2024-09-22T01:17:19.560176+0000","last_clean":"2024-09-22T01:17:19.560176+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:19.560176+0000","last_undegraded":"2024-09-22T01:17:19.560176+0000","last_fullsized":"2024-09-22T01:17:19.560176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.572157+0000","last_change":"2024-09-22T01:17:19.572157+0000","last_active":"2024-09-22T01:17:19.572157+0000","last_peered":"2024-09-22T01:17:19.572157+0000","last_clean":"2024-09-22T01:17:19.572157+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:19.572157+0000","last_undegraded":"2024-09-22T01:17:19.572157+0000","last_fullsized":"2024-09-22T01:17:19.572157+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:12.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":191,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561225+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:19.561225+0000","last_peered":"2024-09-22T01:17:19.561225+0000","last_clean":"2024-09-22T01:17:19.561225+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:19.561225+0000","last_undegraded":"2024-09-22T01:17:19.561225+0000","last_fullsized":"2024-09-22T01:17:19.561225+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561351+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:19.561351+0000","last_peered":"2024-09-22T01:17:19.561351+0000","last_clean":"2024-09-22T01:17:19.561351+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:19.561351+0000","last_undegraded":"2024-09-22T01:17:19.561351+0000","last_fullsized":"2024-09-22T01:17:19.561351+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560076+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:19.560076+0000","last_peered":"2024-09-22T01:17:19.560076+0000","last_clean":"2024-09-22T01:17:19.560076+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:19.560076+0000","last_undegraded":"2024-09-22T01:17:19.560076+0000","last_fullsized":"2024-09-22T01:17:19.560076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3694,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558957+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:19.558957+0000","last_peered":"2024-09-22T01:17:19.558957+0000","last_clean":"2024-09-22T01:17:19.558957+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:19.558957+0000","last_undegraded":"2024-09-22T01:17:19.558957+0000","last_fullsized":"2024-09-22T01:17:19.558957+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169674+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:20.169674+0000","last_peered":"2024-09-22T01:17:20.169674+0000","last_clean":"2024-09-22T01:17:20.169674+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:20.169674+0000","last_undegraded":"2024-09-22T01:17:20.169674+0000","last_fullsized":"2024-09-22T01:17:20.169674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8761,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783798+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:19.783798+0000","last_peered":"2024-09-22T01:17:19.783798+0000","last_clean":"2024-09-22T01:17:19.783798+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:19.783798+0000","last_undegraded":"2024-09-22T01:17:19.783798+0000","last_fullsized":"2024-09-22T01:17:19.783798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":345,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559863+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:19.559863+0000","last_peered":"2024-09-22T01:17:19.559863+0000","last_clean":"2024-09-22T01:17:19.559863+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:19.559863+0000","last_undegraded":"2024-09-22T01:17:19.559863+0000","last_fullsized":"2024-09-22T01:17:19.559863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169382+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:20.169382+0000","last_peered":"2024-09-22T01:17:20.169382+0000","last_clean":"2024-09-22T01:17:20.169382+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:20.169382+0000","last_undegraded":"2024-09-22T01:17:20.169382+0000","last_fullsized":"2024-09-22T01:17:20.169382+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.562928+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:19.562928+0000","last_peered":"2024-09-22T01:17:19.562928+0000","last_clean":"2024-09-22T01:17:19.562928+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:19.562928+0000","last_undegraded":"2024-09-22T01:17:19.562928+0000","last_fullsized":"2024-09-22T01:17:19.562928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7036,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560957+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:19.560957+0000","last_peered":"2024-09-22T01:17:19.560957+0000","last_clean":"2024-09-22T01:17:19.560957+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:19.560957+0000","last_undegraded":"2024-09-22T01:17:19.560957+0000","last_fullsized":"2024-09-22T01:17:19.560957+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559448+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:19.559448+0000","last_peered":"2024-09-22T01:17:19.559448+0000","last_clean":"2024-09-22T01:17:19.559448+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:19.559448+0000","last_undegraded":"2024-09-22T01:17:19.559448+0000","last_fullsized":"2024-09-22T01:17:19.559448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6578,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561235+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:19.561235+0000","last_peered":"2024-09-22T01:17:19.561235+0000","last_clean":"2024-09-22T01:17:19.561235+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:19.561235+0000","last_undegraded":"2024-09-22T01:17:19.561235+0000","last_fullsized":"2024-09-22T01:17:19.561235+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":363,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559762+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:19.559762+0000","last_peered":"2024-09-22T01:17:19.559762+0000","last_clean":"2024-09-22T01:17:19.559762+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:19.559762+0000","last_undegraded":"2024-09-22T01:17:19.559762+0000","last_fullsized":"2024-09-22T01:17:19.559762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":513,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168467+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:20.168467+0000","last_peered":"2024-09-22T01:17:20.168467+0000","last_clean":"2024-09-22T01:17:20.168467+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:20.168467+0000","last_undegraded":"2024-09-22T01:17:20.168467+0000","last_fullsized":"2024-09-22T01:17:20.168467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560902+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:19.560902+0000","last_peered":"2024-09-22T01:17:19.560902+0000","last_clean":"2024-09-22T01:17:19.560902+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:19.560902+0000","last_undegraded":"2024-09-22T01:17:19.560902+0000","last_fullsized":"2024-09-22T01:17:19.560902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19415,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559430+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:19.559430+0000","last_peered":"2024-09-22T01:17:19.559430+0000","last_clean":"2024-09-22T01:17:19.559430+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:19.559430+0000","last_undegraded":"2024-09-22T01:17:19.559430+0000","last_fullsized":"2024-09-22T01:17:19.559430+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4584,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559616+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:19.559616+0000","last_peered":"2024-09-22T01:17:19.559616+0000","last_clean":"2024-09-22T01:17:19.559616+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:19.559616+0000","last_undegraded":"2024-09-22T01:17:19.559616+0000","last_fullsized":"2024-09-22T01:17:19.559616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4970,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.570549+0000","last_change":"2024-09-22T01:17:19.570549+0000","last_active":"2024-09-22T01:17:19.570549+0000","last_peered":"2024-09-22T01:17:19.570549+0000","last_clean":"2024-09-22T01:17:19.570549+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:19.570549+0000","last_undegraded":"2024-09-22T01:17:19.570549+0000","last_fullsized":"2024-09-22T01:17:19.570549+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:27.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3051,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559985+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:19.559985+0000","last_peered":"2024-09-22T01:17:19.559985+0000","last_clean":"2024-09-22T01:17:19.559985+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:19.559985+0000","last_undegraded":"2024-09-22T01:17:19.559985+0000","last_fullsized":"2024-09-22T01:17:19.559985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4151,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783937+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:19.783937+0000","last_peered":"2024-09-22T01:17:19.783937+0000","last_clean":"2024-09-22T01:17:19.783937+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:19.783937+0000","last_undegraded":"2024-09-22T01:17:19.783937+0000","last_fullsized":"2024-09-22T01:17:19.783937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3855,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168348+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:20.168348+0000","last_peered":"2024-09-22T01:17:20.168348+0000","last_clean":"2024-09-22T01:17:20.168348+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:20.168348+0000","last_undegraded":"2024-09-22T01:17:20.168348+0000","last_fullsized":"2024-09-22T01:17:20.168348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5320,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559456+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:19.559456+0000","last_peered":"2024-09-22T01:17:19.559456+0000","last_clean":"2024-09-22T01:17:19.559456+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:19.559456+0000","last_undegraded":"2024-09-22T01:17:19.559456+0000","last_fullsized":"2024-09-22T01:17:19.559456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5318,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558881+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:19.558881+0000","last_peered":"2024-09-22T01:17:19.558881+0000","last_clean":"2024-09-22T01:17:19.558881+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:19.558881+0000","last_undegraded":"2024-09-22T01:17:19.558881+0000","last_fullsized":"2024-09-22T01:17:19.558881+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6046,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559659+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:19.559659+0000","last_peered":"2024-09-22T01:17:19.559659+0000","last_clean":"2024-09-22T01:17:19.559659+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:19.559659+0000","last_undegraded":"2024-09-22T01:17:19.559659+0000","last_fullsized":"2024-09-22T01:17:19.559659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29328,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342512,"statfs":{"total":96636764160,"available":96606732288,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52112,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319728,"statfs":{"total":96636764160,"available":96583401472,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56280,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315560,"statfs":{"total":96636764160,"available":96579133440,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:27.652 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-22T01:17:27.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-22T01:17:27.958 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-22T01:17:27.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-22T01:17:27.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:28.255 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:28.256 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:28.272 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T01:17:27.586238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402300,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572420,"statfs":{"total":773094113280,"available":772682158080,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-11,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.030731"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3321,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169634+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:20.169634+0000","last_peered":"2024-09-22T01:17:20.169634+0000","last_clean":"2024-09-22T01:17:20.169634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:20.169634+0000","last_undegraded":"2024-09-22T01:17:20.169634+0000","last_fullsized":"2024-09-22T01:17:20.169634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12830,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169570+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:20.169570+0000","last_peered":"2024-09-22T01:17:20.169570+0000","last_clean":"2024-09-22T01:17:20.169570+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:20.169570+0000","last_undegraded":"2024-09-22T01:17:20.169570+0000","last_fullsized":"2024-09-22T01:17:20.169570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559885+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:19.559885+0000","last_peered":"2024-09-22T01:17:19.559885+0000","last_clean":"2024-09-22T01:17:19.559885+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:19.559885+0000","last_undegraded":"2024-09-22T01:17:19.559885+0000","last_fullsized":"2024-09-22T01:17:19.559885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168447+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:20.168447+0000","last_peered":"2024-09-22T01:17:20.168447+0000","last_clean":"2024-09-22T01:17:20.168447+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:20.168447+0000","last_undegraded":"2024-09-22T01:17:20.168447+0000","last_fullsized":"2024-09-22T01:17:20.168447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561110+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:19.561110+0000","last_peered":"2024-09-22T01:17:19.561110+0000","last_clean":"2024-09-22T01:17:19.561110+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:19.561110+0000","last_undegraded":"2024-09-22T01:17:19.561110+0000","last_fullsized":"2024-09-22T01:17:19.561110+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17421,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168499+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:20.168499+0000","last_peered":"2024-09-22T01:17:20.168499+0000","last_clean":"2024-09-22T01:17:20.168499+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:20.168499+0000","last_undegraded":"2024-09-22T01:17:20.168499+0000","last_fullsized":"2024-09-22T01:17:20.168499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6612,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560970+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:19.560970+0000","last_peered":"2024-09-22T01:17:19.560970+0000","last_clean":"2024-09-22T01:17:19.560970+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:19.560970+0000","last_undegraded":"2024-09-22T01:17:19.560970+0000","last_fullsized":"2024-09-22T01:17:19.560970+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169725+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:20.169725+0000","last_peered":"2024-09-22T01:17:20.169725+0000","last_clean":"2024-09-22T01:17:20.169725+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:20.169725+0000","last_undegraded":"2024-09-22T01:17:20.169725+0000","last_fullsized":"2024-09-22T01:17:20.169725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5802,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169776+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:20.169776+0000","last_peered":"2024-09-22T01:17:20.169776+0000","last_clean":"2024-09-22T01:17:20.169776+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:20.169776+0000","last_undegraded":"2024-09-22T01:17:20.169776+0000","last_fullsized":"2024-09-22T01:17:20.169776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13509,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783820+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:19.783820+0000","last_peered":"2024-09-22T01:17:19.783820+0000","last_clean":"2024-09-22T01:17:19.783820+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:19.783820+0000","last_undegraded":"2024-09-22T01:17:19.783820+0000","last_fullsized":"2024-09-22T01:17:19.783820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10661,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559715+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:19.559715+0000","last_peered":"2024-09-22T01:17:19.559715+0000","last_clean":"2024-09-22T01:17:19.559715+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:19.559715+0000","last_undegraded":"2024-09-22T01:17:19.559715+0000","last_fullsized":"2024-09-22T01:17:19.559715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560176+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:19.560176+0000","last_peered":"2024-09-22T01:17:19.560176+0000","last_clean":"2024-09-22T01:17:19.560176+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:19.560176+0000","last_undegraded":"2024-09-22T01:17:19.560176+0000","last_fullsized":"2024-09-22T01:17:19.560176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.572157+0000","last_change":"2024-09-22T01:17:19.572157+0000","last_active":"2024-09-22T01:17:19.572157+0000","last_peered":"2024-09-22T01:17:19.572157+0000","last_clean":"2024-09-22T01:17:19.572157+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:19.572157+0000","last_undegraded":"2024-09-22T01:17:19.572157+0000","last_fullsized":"2024-09-22T01:17:19.572157+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:12.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":191,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561225+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:19.561225+0000","last_peered":"2024-09-22T01:17:19.561225+0000","last_clean":"2024-09-22T01:17:19.561225+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:19.561225+0000","last_undegraded":"2024-09-22T01:17:19.561225+0000","last_fullsized":"2024-09-22T01:17:19.561225+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561351+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:19.561351+0000","last_peered":"2024-09-22T01:17:19.561351+0000","last_clean":"2024-09-22T01:17:19.561351+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:19.561351+0000","last_undegraded":"2024-09-22T01:17:19.561351+0000","last_fullsized":"2024-09-22T01:17:19.561351+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560076+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:19.560076+0000","last_peered":"2024-09-22T01:17:19.560076+0000","last_clean":"2024-09-22T01:17:19.560076+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:19.560076+0000","last_undegraded":"2024-09-22T01:17:19.560076+0000","last_fullsized":"2024-09-22T01:17:19.560076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3694,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558957+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:19.558957+0000","last_peered":"2024-09-22T01:17:19.558957+0000","last_clean":"2024-09-22T01:17:19.558957+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:19.558957+0000","last_undegraded":"2024-09-22T01:17:19.558957+0000","last_fullsized":"2024-09-22T01:17:19.558957+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169674+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:20.169674+0000","last_peered":"2024-09-22T01:17:20.169674+0000","last_clean":"2024-09-22T01:17:20.169674+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:20.169674+0000","last_undegraded":"2024-09-22T01:17:20.169674+0000","last_fullsized":"2024-09-22T01:17:20.169674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8761,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783798+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:19.783798+0000","last_peered":"2024-09-22T01:17:19.783798+0000","last_clean":"2024-09-22T01:17:19.783798+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:19.783798+0000","last_undegraded":"2024-09-22T01:17:19.783798+0000","last_fullsized":"2024-09-22T01:17:19.783798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":345,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559863+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:19.559863+0000","last_peered":"2024-09-22T01:17:19.559863+0000","last_clean":"2024-09-22T01:17:19.559863+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:19.559863+0000","last_undegraded":"2024-09-22T01:17:19.559863+0000","last_fullsized":"2024-09-22T01:17:19.559863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169382+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:20.169382+0000","last_peered":"2024-09-22T01:17:20.169382+0000","last_clean":"2024-09-22T01:17:20.169382+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:20.169382+0000","last_undegraded":"2024-09-22T01:17:20.169382+0000","last_fullsized":"2024-09-22T01:17:20.169382+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.562928+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:19.562928+0000","last_peered":"2024-09-22T01:17:19.562928+0000","last_clean":"2024-09-22T01:17:19.562928+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:19.562928+0000","last_undegraded":"2024-09-22T01:17:19.562928+0000","last_fullsized":"2024-09-22T01:17:19.562928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7036,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560957+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:19.560957+0000","last_peered":"2024-09-22T01:17:19.560957+0000","last_clean":"2024-09-22T01:17:19.560957+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:19.560957+0000","last_undegraded":"2024-09-22T01:17:19.560957+0000","last_fullsized":"2024-09-22T01:17:19.560957+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559448+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:19.559448+0000","last_peered":"2024-09-22T01:17:19.559448+0000","last_clean":"2024-09-22T01:17:19.559448+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:19.559448+0000","last_undegraded":"2024-09-22T01:17:19.559448+0000","last_fullsized":"2024-09-22T01:17:19.559448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6578,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561235+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:19.561235+0000","last_peered":"2024-09-22T01:17:19.561235+0000","last_clean":"2024-09-22T01:17:19.561235+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:19.561235+0000","last_undegraded":"2024-09-22T01:17:19.561235+0000","last_fullsized":"2024-09-22T01:17:19.561235+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":363,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559762+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:19.559762+0000","last_peered":"2024-09-22T01:17:19.559762+0000","last_clean":"2024-09-22T01:17:19.559762+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:19.559762+0000","last_undegraded":"2024-09-22T01:17:19.559762+0000","last_fullsized":"2024-09-22T01:17:19.559762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":513,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168467+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:20.168467+0000","last_peered":"2024-09-22T01:17:20.168467+0000","last_clean":"2024-09-22T01:17:20.168467+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:20.168467+0000","last_undegraded":"2024-09-22T01:17:20.168467+0000","last_fullsized":"2024-09-22T01:17:20.168467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560902+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:19.560902+0000","last_peered":"2024-09-22T01:17:19.560902+0000","last_clean":"2024-09-22T01:17:19.560902+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:19.560902+0000","last_undegraded":"2024-09-22T01:17:19.560902+0000","last_fullsized":"2024-09-22T01:17:19.560902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19415,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559430+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:19.559430+0000","last_peered":"2024-09-22T01:17:19.559430+0000","last_clean":"2024-09-22T01:17:19.559430+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:19.559430+0000","last_undegraded":"2024-09-22T01:17:19.559430+0000","last_fullsized":"2024-09-22T01:17:19.559430+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4584,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559616+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:19.559616+0000","last_peered":"2024-09-22T01:17:19.559616+0000","last_clean":"2024-09-22T01:17:19.559616+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:19.559616+0000","last_undegraded":"2024-09-22T01:17:19.559616+0000","last_fullsized":"2024-09-22T01:17:19.559616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4970,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.570549+0000","last_change":"2024-09-22T01:17:19.570549+0000","last_active":"2024-09-22T01:17:19.570549+0000","last_peered":"2024-09-22T01:17:19.570549+0000","last_clean":"2024-09-22T01:17:19.570549+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:19.570549+0000","last_undegraded":"2024-09-22T01:17:19.570549+0000","last_fullsized":"2024-09-22T01:17:19.570549+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:27.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3051,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559985+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:19.559985+0000","last_peered":"2024-09-22T01:17:19.559985+0000","last_clean":"2024-09-22T01:17:19.559985+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:19.559985+0000","last_undegraded":"2024-09-22T01:17:19.559985+0000","last_fullsized":"2024-09-22T01:17:19.559985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4151,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783937+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:19.783937+0000","last_peered":"2024-09-22T01:17:19.783937+0000","last_clean":"2024-09-22T01:17:19.783937+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:19.783937+0000","last_undegraded":"2024-09-22T01:17:19.783937+0000","last_fullsized":"2024-09-22T01:17:19.783937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3855,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168348+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:20.168348+0000","last_peered":"2024-09-22T01:17:20.168348+0000","last_clean":"2024-09-22T01:17:20.168348+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:20.168348+0000","last_undegraded":"2024-09-22T01:17:20.168348+0000","last_fullsized":"2024-09-22T01:17:20.168348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5320,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559456+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:19.559456+0000","last_peered":"2024-09-22T01:17:19.559456+0000","last_clean":"2024-09-22T01:17:19.559456+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:19.559456+0000","last_undegraded":"2024-09-22T01:17:19.559456+0000","last_fullsized":"2024-09-22T01:17:19.559456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5318,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558881+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:19.558881+0000","last_peered":"2024-09-22T01:17:19.558881+0000","last_clean":"2024-09-22T01:17:19.558881+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:19.558881+0000","last_undegraded":"2024-09-22T01:17:19.558881+0000","last_fullsized":"2024-09-22T01:17:19.558881+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6046,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559659+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:19.559659+0000","last_peered":"2024-09-22T01:17:19.559659+0000","last_clean":"2024-09-22T01:17:19.559659+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:19.559659+0000","last_undegraded":"2024-09-22T01:17:19.559659+0000","last_fullsized":"2024-09-22T01:17:19.559659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29328,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342512,"statfs":{"total":96636764160,"available":96606732288,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52112,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319728,"statfs":{"total":96636764160,"available":96583401472,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56280,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315560,"statfs":{"total":96636764160,"available":96579133440,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:28.274 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:28.559 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:28.559 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:28.577 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T01:17:27.586238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402300,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572420,"statfs":{"total":773094113280,"available":772682158080,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-11,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.030731"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3321,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169634+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:20.169634+0000","last_peered":"2024-09-22T01:17:20.169634+0000","last_clean":"2024-09-22T01:17:20.169634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:20.169634+0000","last_undegraded":"2024-09-22T01:17:20.169634+0000","last_fullsized":"2024-09-22T01:17:20.169634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12830,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169570+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:20.169570+0000","last_peered":"2024-09-22T01:17:20.169570+0000","last_clean":"2024-09-22T01:17:20.169570+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:20.169570+0000","last_undegraded":"2024-09-22T01:17:20.169570+0000","last_fullsized":"2024-09-22T01:17:20.169570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559885+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:19.559885+0000","last_peered":"2024-09-22T01:17:19.559885+0000","last_clean":"2024-09-22T01:17:19.559885+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:19.559885+0000","last_undegraded":"2024-09-22T01:17:19.559885+0000","last_fullsized":"2024-09-22T01:17:19.559885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168447+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:20.168447+0000","last_peered":"2024-09-22T01:17:20.168447+0000","last_clean":"2024-09-22T01:17:20.168447+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:20.168447+0000","last_undegraded":"2024-09-22T01:17:20.168447+0000","last_fullsized":"2024-09-22T01:17:20.168447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561110+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:19.561110+0000","last_peered":"2024-09-22T01:17:19.561110+0000","last_clean":"2024-09-22T01:17:19.561110+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:19.561110+0000","last_undegraded":"2024-09-22T01:17:19.561110+0000","last_fullsized":"2024-09-22T01:17:19.561110+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17421,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168499+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:20.168499+0000","last_peered":"2024-09-22T01:17:20.168499+0000","last_clean":"2024-09-22T01:17:20.168499+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:20.168499+0000","last_undegraded":"2024-09-22T01:17:20.168499+0000","last_fullsized":"2024-09-22T01:17:20.168499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6612,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560970+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:19.560970+0000","last_peered":"2024-09-22T01:17:19.560970+0000","last_clean":"2024-09-22T01:17:19.560970+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:19.560970+0000","last_undegraded":"2024-09-22T01:17:19.560970+0000","last_fullsized":"2024-09-22T01:17:19.560970+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169725+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:20.169725+0000","last_peered":"2024-09-22T01:17:20.169725+0000","last_clean":"2024-09-22T01:17:20.169725+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:20.169725+0000","last_undegraded":"2024-09-22T01:17:20.169725+0000","last_fullsized":"2024-09-22T01:17:20.169725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5802,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169776+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:20.169776+0000","last_peered":"2024-09-22T01:17:20.169776+0000","last_clean":"2024-09-22T01:17:20.169776+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:20.169776+0000","last_undegraded":"2024-09-22T01:17:20.169776+0000","last_fullsized":"2024-09-22T01:17:20.169776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13509,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783820+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:19.783820+0000","last_peered":"2024-09-22T01:17:19.783820+0000","last_clean":"2024-09-22T01:17:19.783820+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:19.783820+0000","last_undegraded":"2024-09-22T01:17:19.783820+0000","last_fullsized":"2024-09-22T01:17:19.783820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10661,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559715+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:19.559715+0000","last_peered":"2024-09-22T01:17:19.559715+0000","last_clean":"2024-09-22T01:17:19.559715+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:19.559715+0000","last_undegraded":"2024-09-22T01:17:19.559715+0000","last_fullsized":"2024-09-22T01:17:19.559715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560176+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:19.560176+0000","last_peered":"2024-09-22T01:17:19.560176+0000","last_clean":"2024-09-22T01:17:19.560176+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:19.560176+0000","last_undegraded":"2024-09-22T01:17:19.560176+0000","last_fullsized":"2024-09-22T01:17:19.560176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.572157+0000","last_change":"2024-09-22T01:17:19.572157+0000","last_active":"2024-09-22T01:17:19.572157+0000","last_peered":"2024-09-22T01:17:19.572157+0000","last_clean":"2024-09-22T01:17:19.572157+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:19.572157+0000","last_undegraded":"2024-09-22T01:17:19.572157+0000","last_fullsized":"2024-09-22T01:17:19.572157+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:12.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":191,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561225+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:19.561225+0000","last_peered":"2024-09-22T01:17:19.561225+0000","last_clean":"2024-09-22T01:17:19.561225+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:19.561225+0000","last_undegraded":"2024-09-22T01:17:19.561225+0000","last_fullsized":"2024-09-22T01:17:19.561225+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561351+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:19.561351+0000","last_peered":"2024-09-22T01:17:19.561351+0000","last_clean":"2024-09-22T01:17:19.561351+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:19.561351+0000","last_undegraded":"2024-09-22T01:17:19.561351+0000","last_fullsized":"2024-09-22T01:17:19.561351+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560076+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:19.560076+0000","last_peered":"2024-09-22T01:17:19.560076+0000","last_clean":"2024-09-22T01:17:19.560076+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:19.560076+0000","last_undegraded":"2024-09-22T01:17:19.560076+0000","last_fullsized":"2024-09-22T01:17:19.560076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3694,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558957+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:19.558957+0000","last_peered":"2024-09-22T01:17:19.558957+0000","last_clean":"2024-09-22T01:17:19.558957+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:19.558957+0000","last_undegraded":"2024-09-22T01:17:19.558957+0000","last_fullsized":"2024-09-22T01:17:19.558957+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169674+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:20.169674+0000","last_peered":"2024-09-22T01:17:20.169674+0000","last_clean":"2024-09-22T01:17:20.169674+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:20.169674+0000","last_undegraded":"2024-09-22T01:17:20.169674+0000","last_fullsized":"2024-09-22T01:17:20.169674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8761,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783798+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:19.783798+0000","last_peered":"2024-09-22T01:17:19.783798+0000","last_clean":"2024-09-22T01:17:19.783798+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:19.783798+0000","last_undegraded":"2024-09-22T01:17:19.783798+0000","last_fullsized":"2024-09-22T01:17:19.783798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":345,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559863+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:19.559863+0000","last_peered":"2024-09-22T01:17:19.559863+0000","last_clean":"2024-09-22T01:17:19.559863+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:19.559863+0000","last_undegraded":"2024-09-22T01:17:19.559863+0000","last_fullsized":"2024-09-22T01:17:19.559863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169382+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:20.169382+0000","last_peered":"2024-09-22T01:17:20.169382+0000","last_clean":"2024-09-22T01:17:20.169382+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:20.169382+0000","last_undegraded":"2024-09-22T01:17:20.169382+0000","last_fullsized":"2024-09-22T01:17:20.169382+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.562928+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:19.562928+0000","last_peered":"2024-09-22T01:17:19.562928+0000","last_clean":"2024-09-22T01:17:19.562928+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:19.562928+0000","last_undegraded":"2024-09-22T01:17:19.562928+0000","last_fullsized":"2024-09-22T01:17:19.562928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7036,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560957+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:19.560957+0000","last_peered":"2024-09-22T01:17:19.560957+0000","last_clean":"2024-09-22T01:17:19.560957+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:19.560957+0000","last_undegraded":"2024-09-22T01:17:19.560957+0000","last_fullsized":"2024-09-22T01:17:19.560957+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559448+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:19.559448+0000","last_peered":"2024-09-22T01:17:19.559448+0000","last_clean":"2024-09-22T01:17:19.559448+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:19.559448+0000","last_undegraded":"2024-09-22T01:17:19.559448+0000","last_fullsized":"2024-09-22T01:17:19.559448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6578,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561235+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:19.561235+0000","last_peered":"2024-09-22T01:17:19.561235+0000","last_clean":"2024-09-22T01:17:19.561235+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:19.561235+0000","last_undegraded":"2024-09-22T01:17:19.561235+0000","last_fullsized":"2024-09-22T01:17:19.561235+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":363,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559762+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:19.559762+0000","last_peered":"2024-09-22T01:17:19.559762+0000","last_clean":"2024-09-22T01:17:19.559762+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:19.559762+0000","last_undegraded":"2024-09-22T01:17:19.559762+0000","last_fullsized":"2024-09-22T01:17:19.559762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":513,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168467+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:20.168467+0000","last_peered":"2024-09-22T01:17:20.168467+0000","last_clean":"2024-09-22T01:17:20.168467+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:20.168467+0000","last_undegraded":"2024-09-22T01:17:20.168467+0000","last_fullsized":"2024-09-22T01:17:20.168467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560902+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:19.560902+0000","last_peered":"2024-09-22T01:17:19.560902+0000","last_clean":"2024-09-22T01:17:19.560902+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:19.560902+0000","last_undegraded":"2024-09-22T01:17:19.560902+0000","last_fullsized":"2024-09-22T01:17:19.560902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19415,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559430+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:19.559430+0000","last_peered":"2024-09-22T01:17:19.559430+0000","last_clean":"2024-09-22T01:17:19.559430+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:19.559430+0000","last_undegraded":"2024-09-22T01:17:19.559430+0000","last_fullsized":"2024-09-22T01:17:19.559430+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4584,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559616+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:19.559616+0000","last_peered":"2024-09-22T01:17:19.559616+0000","last_clean":"2024-09-22T01:17:19.559616+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:19.559616+0000","last_undegraded":"2024-09-22T01:17:19.559616+0000","last_fullsized":"2024-09-22T01:17:19.559616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4970,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.570549+0000","last_change":"2024-09-22T01:17:19.570549+0000","last_active":"2024-09-22T01:17:19.570549+0000","last_peered":"2024-09-22T01:17:19.570549+0000","last_clean":"2024-09-22T01:17:19.570549+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:19.570549+0000","last_undegraded":"2024-09-22T01:17:19.570549+0000","last_fullsized":"2024-09-22T01:17:19.570549+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:27.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3051,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559985+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:19.559985+0000","last_peered":"2024-09-22T01:17:19.559985+0000","last_clean":"2024-09-22T01:17:19.559985+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:19.559985+0000","last_undegraded":"2024-09-22T01:17:19.559985+0000","last_fullsized":"2024-09-22T01:17:19.559985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4151,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783937+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:19.783937+0000","last_peered":"2024-09-22T01:17:19.783937+0000","last_clean":"2024-09-22T01:17:19.783937+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:19.783937+0000","last_undegraded":"2024-09-22T01:17:19.783937+0000","last_fullsized":"2024-09-22T01:17:19.783937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3855,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168348+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:20.168348+0000","last_peered":"2024-09-22T01:17:20.168348+0000","last_clean":"2024-09-22T01:17:20.168348+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:20.168348+0000","last_undegraded":"2024-09-22T01:17:20.168348+0000","last_fullsized":"2024-09-22T01:17:20.168348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5320,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559456+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:19.559456+0000","last_peered":"2024-09-22T01:17:19.559456+0000","last_clean":"2024-09-22T01:17:19.559456+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:19.559456+0000","last_undegraded":"2024-09-22T01:17:19.559456+0000","last_fullsized":"2024-09-22T01:17:19.559456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5318,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558881+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:19.558881+0000","last_peered":"2024-09-22T01:17:19.558881+0000","last_clean":"2024-09-22T01:17:19.558881+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:19.558881+0000","last_undegraded":"2024-09-22T01:17:19.558881+0000","last_fullsized":"2024-09-22T01:17:19.558881+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6046,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559659+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:19.559659+0000","last_peered":"2024-09-22T01:17:19.559659+0000","last_clean":"2024-09-22T01:17:19.559659+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:19.559659+0000","last_undegraded":"2024-09-22T01:17:19.559659+0000","last_fullsized":"2024-09-22T01:17:19.559659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29328,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342512,"statfs":{"total":96636764160,"available":96606732288,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52112,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319728,"statfs":{"total":96636764160,"available":96583401472,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56280,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315560,"statfs":{"total":96636764160,"available":96579133440,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:28.579 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-22T01:17:28.579 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:28.859 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:28.860 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:28.876 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-22T01:17:27.586238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21984,"ondisk_log_size":21984,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402300,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572420,"statfs":{"total":773094113280,"available":772682158080,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-11,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":523,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.030731"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3321,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169634+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:20.169634+0000","last_peered":"2024-09-22T01:17:20.169634+0000","last_clean":"2024-09-22T01:17:20.169634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:20.169634+0000","last_undegraded":"2024-09-22T01:17:20.169634+0000","last_fullsized":"2024-09-22T01:17:20.169634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12830,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169570+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:20.169570+0000","last_peered":"2024-09-22T01:17:20.169570+0000","last_clean":"2024-09-22T01:17:20.169570+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:20.169570+0000","last_undegraded":"2024-09-22T01:17:20.169570+0000","last_fullsized":"2024-09-22T01:17:20.169570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559885+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:19.559885+0000","last_peered":"2024-09-22T01:17:19.559885+0000","last_clean":"2024-09-22T01:17:19.559885+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:19.559885+0000","last_undegraded":"2024-09-22T01:17:19.559885+0000","last_fullsized":"2024-09-22T01:17:19.559885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168447+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:20.168447+0000","last_peered":"2024-09-22T01:17:20.168447+0000","last_clean":"2024-09-22T01:17:20.168447+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:20.168447+0000","last_undegraded":"2024-09-22T01:17:20.168447+0000","last_fullsized":"2024-09-22T01:17:20.168447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561110+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:19.561110+0000","last_peered":"2024-09-22T01:17:19.561110+0000","last_clean":"2024-09-22T01:17:19.561110+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:19.561110+0000","last_undegraded":"2024-09-22T01:17:19.561110+0000","last_fullsized":"2024-09-22T01:17:19.561110+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17421,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168499+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:20.168499+0000","last_peered":"2024-09-22T01:17:20.168499+0000","last_clean":"2024-09-22T01:17:20.168499+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:20.168499+0000","last_undegraded":"2024-09-22T01:17:20.168499+0000","last_fullsized":"2024-09-22T01:17:20.168499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6612,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560970+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:19.560970+0000","last_peered":"2024-09-22T01:17:19.560970+0000","last_clean":"2024-09-22T01:17:19.560970+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:19.560970+0000","last_undegraded":"2024-09-22T01:17:19.560970+0000","last_fullsized":"2024-09-22T01:17:19.560970+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169725+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:20.169725+0000","last_peered":"2024-09-22T01:17:20.169725+0000","last_clean":"2024-09-22T01:17:20.169725+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:20.169725+0000","last_undegraded":"2024-09-22T01:17:20.169725+0000","last_fullsized":"2024-09-22T01:17:20.169725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5802,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169776+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:20.169776+0000","last_peered":"2024-09-22T01:17:20.169776+0000","last_clean":"2024-09-22T01:17:20.169776+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:20.169776+0000","last_undegraded":"2024-09-22T01:17:20.169776+0000","last_fullsized":"2024-09-22T01:17:20.169776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13509,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783820+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:19.783820+0000","last_peered":"2024-09-22T01:17:19.783820+0000","last_clean":"2024-09-22T01:17:19.783820+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:19.783820+0000","last_undegraded":"2024-09-22T01:17:19.783820+0000","last_fullsized":"2024-09-22T01:17:19.783820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"168'851","reported_seq":4126,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168676+0000","last_change":"2024-09-22T01:16:52.673528+0000","last_active":"2024-09-22T01:17:20.168676+0000","last_peered":"2024-09-22T01:17:20.168676+0000","last_clean":"2024-09-22T01:17:20.168676+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:20.168676+0000","last_undegraded":"2024-09-22T01:17:20.168676+0000","last_fullsized":"2024-09-22T01:17:20.168676+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:58:42.422170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10661,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559715+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:19.559715+0000","last_peered":"2024-09-22T01:17:19.559715+0000","last_clean":"2024-09-22T01:17:19.559715+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:19.559715+0000","last_undegraded":"2024-09-22T01:17:19.559715+0000","last_fullsized":"2024-09-22T01:17:19.559715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560176+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:19.560176+0000","last_peered":"2024-09-22T01:17:19.560176+0000","last_clean":"2024-09-22T01:17:19.560176+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:19.560176+0000","last_undegraded":"2024-09-22T01:17:19.560176+0000","last_fullsized":"2024-09-22T01:17:19.560176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.572157+0000","last_change":"2024-09-22T01:17:19.572157+0000","last_active":"2024-09-22T01:17:19.572157+0000","last_peered":"2024-09-22T01:17:19.572157+0000","last_clean":"2024-09-22T01:17:19.572157+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:19.572157+0000","last_undegraded":"2024-09-22T01:17:19.572157+0000","last_fullsized":"2024-09-22T01:17:19.572157+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:12.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":191,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561225+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:19.561225+0000","last_peered":"2024-09-22T01:17:19.561225+0000","last_clean":"2024-09-22T01:17:19.561225+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:19.561225+0000","last_undegraded":"2024-09-22T01:17:19.561225+0000","last_fullsized":"2024-09-22T01:17:19.561225+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561351+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:19.561351+0000","last_peered":"2024-09-22T01:17:19.561351+0000","last_clean":"2024-09-22T01:17:19.561351+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:19.561351+0000","last_undegraded":"2024-09-22T01:17:19.561351+0000","last_fullsized":"2024-09-22T01:17:19.561351+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560076+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:19.560076+0000","last_peered":"2024-09-22T01:17:19.560076+0000","last_clean":"2024-09-22T01:17:19.560076+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:19.560076+0000","last_undegraded":"2024-09-22T01:17:19.560076+0000","last_fullsized":"2024-09-22T01:17:19.560076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3694,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558957+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:19.558957+0000","last_peered":"2024-09-22T01:17:19.558957+0000","last_clean":"2024-09-22T01:17:19.558957+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:19.558957+0000","last_undegraded":"2024-09-22T01:17:19.558957+0000","last_fullsized":"2024-09-22T01:17:19.558957+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169674+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:20.169674+0000","last_peered":"2024-09-22T01:17:20.169674+0000","last_clean":"2024-09-22T01:17:20.169674+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:20.169674+0000","last_undegraded":"2024-09-22T01:17:20.169674+0000","last_fullsized":"2024-09-22T01:17:20.169674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8761,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783798+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:19.783798+0000","last_peered":"2024-09-22T01:17:19.783798+0000","last_clean":"2024-09-22T01:17:19.783798+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:19.783798+0000","last_undegraded":"2024-09-22T01:17:19.783798+0000","last_fullsized":"2024-09-22T01:17:19.783798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6982,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168992+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.168992+0000","last_peered":"2024-09-22T01:17:20.168992+0000","last_clean":"2024-09-22T01:17:20.168992+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.168992+0000","last_undegraded":"2024-09-22T01:17:20.168992+0000","last_fullsized":"2024-09-22T01:17:20.168992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:07:17.059550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":345,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559863+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:19.559863+0000","last_peered":"2024-09-22T01:17:19.559863+0000","last_clean":"2024-09-22T01:17:19.559863+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:19.559863+0000","last_undegraded":"2024-09-22T01:17:19.559863+0000","last_fullsized":"2024-09-22T01:17:19.559863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169382+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:20.169382+0000","last_peered":"2024-09-22T01:17:20.169382+0000","last_clean":"2024-09-22T01:17:20.169382+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:20.169382+0000","last_undegraded":"2024-09-22T01:17:20.169382+0000","last_fullsized":"2024-09-22T01:17:20.169382+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.562928+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:19.562928+0000","last_peered":"2024-09-22T01:17:19.562928+0000","last_clean":"2024-09-22T01:17:19.562928+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:19.562928+0000","last_undegraded":"2024-09-22T01:17:19.562928+0000","last_fullsized":"2024-09-22T01:17:19.562928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7036,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560957+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:19.560957+0000","last_peered":"2024-09-22T01:17:19.560957+0000","last_clean":"2024-09-22T01:17:19.560957+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:19.560957+0000","last_undegraded":"2024-09-22T01:17:19.560957+0000","last_fullsized":"2024-09-22T01:17:19.560957+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559448+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:19.559448+0000","last_peered":"2024-09-22T01:17:19.559448+0000","last_clean":"2024-09-22T01:17:19.559448+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:19.559448+0000","last_undegraded":"2024-09-22T01:17:19.559448+0000","last_fullsized":"2024-09-22T01:17:19.559448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6578,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561235+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:19.561235+0000","last_peered":"2024-09-22T01:17:19.561235+0000","last_clean":"2024-09-22T01:17:19.561235+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:19.561235+0000","last_undegraded":"2024-09-22T01:17:19.561235+0000","last_fullsized":"2024-09-22T01:17:19.561235+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":363,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559762+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:19.559762+0000","last_peered":"2024-09-22T01:17:19.559762+0000","last_clean":"2024-09-22T01:17:19.559762+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:19.559762+0000","last_undegraded":"2024-09-22T01:17:19.559762+0000","last_fullsized":"2024-09-22T01:17:19.559762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":513,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168467+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:20.168467+0000","last_peered":"2024-09-22T01:17:20.168467+0000","last_clean":"2024-09-22T01:17:20.168467+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:20.168467+0000","last_undegraded":"2024-09-22T01:17:20.168467+0000","last_fullsized":"2024-09-22T01:17:20.168467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560902+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:19.560902+0000","last_peered":"2024-09-22T01:17:19.560902+0000","last_clean":"2024-09-22T01:17:19.560902+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:19.560902+0000","last_undegraded":"2024-09-22T01:17:19.560902+0000","last_fullsized":"2024-09-22T01:17:19.560902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19415,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559430+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:19.559430+0000","last_peered":"2024-09-22T01:17:19.559430+0000","last_clean":"2024-09-22T01:17:19.559430+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:19.559430+0000","last_undegraded":"2024-09-22T01:17:19.559430+0000","last_fullsized":"2024-09-22T01:17:19.559430+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4584,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559616+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:19.559616+0000","last_peered":"2024-09-22T01:17:19.559616+0000","last_clean":"2024-09-22T01:17:19.559616+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:19.559616+0000","last_undegraded":"2024-09-22T01:17:19.559616+0000","last_fullsized":"2024-09-22T01:17:19.559616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4970,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.570549+0000","last_change":"2024-09-22T01:17:19.570549+0000","last_active":"2024-09-22T01:17:19.570549+0000","last_peered":"2024-09-22T01:17:19.570549+0000","last_clean":"2024-09-22T01:17:19.570549+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:19.570549+0000","last_undegraded":"2024-09-22T01:17:19.570549+0000","last_fullsized":"2024-09-22T01:17:19.570549+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:27.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6326,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168472+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168472+0000","last_peered":"2024-09-22T01:17:20.168472+0000","last_clean":"2024-09-22T01:17:20.168472+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168472+0000","last_undegraded":"2024-09-22T01:17:20.168472+0000","last_fullsized":"2024-09-22T01:17:20.168472+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:33:54.790124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3051,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559985+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:19.559985+0000","last_peered":"2024-09-22T01:17:19.559985+0000","last_clean":"2024-09-22T01:17:19.559985+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:19.559985+0000","last_undegraded":"2024-09-22T01:17:19.559985+0000","last_fullsized":"2024-09-22T01:17:19.559985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4151,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783937+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:19.783937+0000","last_peered":"2024-09-22T01:17:19.783937+0000","last_clean":"2024-09-22T01:17:19.783937+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:19.783937+0000","last_undegraded":"2024-09-22T01:17:19.783937+0000","last_fullsized":"2024-09-22T01:17:19.783937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"170'713","reported_seq":4103,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168720+0000","last_change":"2024-09-22T01:16:52.674558+0000","last_active":"2024-09-22T01:17:20.168720+0000","last_peered":"2024-09-22T01:17:20.168720+0000","last_clean":"2024-09-22T01:17:20.168720+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:20.168720+0000","last_undegraded":"2024-09-22T01:17:20.168720+0000","last_fullsized":"2024-09-22T01:17:20.168720+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:39.077592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3855,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168348+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:20.168348+0000","last_peered":"2024-09-22T01:17:20.168348+0000","last_clean":"2024-09-22T01:17:20.168348+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:20.168348+0000","last_undegraded":"2024-09-22T01:17:20.168348+0000","last_fullsized":"2024-09-22T01:17:20.168348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5320,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559456+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:19.559456+0000","last_peered":"2024-09-22T01:17:19.559456+0000","last_clean":"2024-09-22T01:17:19.559456+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:19.559456+0000","last_undegraded":"2024-09-22T01:17:19.559456+0000","last_fullsized":"2024-09-22T01:17:19.559456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5318,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558881+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:19.558881+0000","last_peered":"2024-09-22T01:17:19.558881+0000","last_clean":"2024-09-22T01:17:19.558881+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:19.558881+0000","last_undegraded":"2024-09-22T01:17:19.558881+0000","last_fullsized":"2024-09-22T01:17:19.558881+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6046,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559659+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:19.559659+0000","last_peered":"2024-09-22T01:17:19.559659+0000","last_clean":"2024-09-22T01:17:19.559659+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:19.559659+0000","last_undegraded":"2024-09-22T01:17:19.559659+0000","last_fullsized":"2024-09-22T01:17:19.559659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21864,"ondisk_log_size":21864,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29328,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342512,"statfs":{"total":96636764160,"available":96606732288,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52112,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319728,"statfs":{"total":96636764160,"available":96583401472,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56280,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315560,"statfs":{"total":96636764160,"available":96579133440,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:28.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-22T01:17:29.156 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:29.156 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":195,"fsid":"e212681a-0600-4f32-ab96-8ca46d37e9a4","created":"2024-09-22T01:13:34.805751+0000","modified":"2024-09-22T01:17:19.552531+0000","last_up_change":"2024-09-22T01:13:47.939773+0000","last_in_change":"2024-09-22T01:14:37.510453+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-22T01:13:49.594720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-22T01:13:53.081727+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-22T01:14:03.263074+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":54,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a0e12724-c6a8-43ce-aad2-d3043c2f23d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6801","nonce":3838492416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6803","nonce":3838492416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6807","nonce":3838492416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":3838492416},{"type":"v1","addr":"172.21.15.96:6805","nonce":3838492416}]},"public_addr":"172.21.15.96:6801/3838492416","cluster_addr":"172.21.15.96:6803/3838492416","heartbeat_back_addr":"172.21.15.96:6807/3838492416","heartbeat_front_addr":"172.21.15.96:6805/3838492416","state":["exists","up"]},{"osd":1,"uuid":"bd73805d-596e-40f0-ba54-69ca84175c51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6825","nonce":1053715553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6827","nonce":1053715553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6831","nonce":1053715553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1053715553},{"type":"v1","addr":"172.21.15.96:6829","nonce":1053715553}]},"public_addr":"172.21.15.96:6825/1053715553","cluster_addr":"172.21.15.96:6827/1053715553","heartbeat_back_addr":"172.21.15.96:6831/1053715553","heartbeat_front_addr":"172.21.15.96:6829/1053715553","state":["exists","up"]},{"osd":2,"uuid":"90cf3d2b-dc6e-4ee0-8663-7c62e722e7b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6817","nonce":860290285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6819","nonce":860290285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6823","nonce":860290285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":860290285},{"type":"v1","addr":"172.21.15.96:6821","nonce":860290285}]},"public_addr":"172.21.15.96:6817/860290285","cluster_addr":"172.21.15.96:6819/860290285","heartbeat_back_addr":"172.21.15.96:6823/860290285","heartbeat_front_addr":"172.21.15.96:6821/860290285","state":["exists","up"]},{"osd":3,"uuid":"eed7a22f-aa64-4dad-9151-8fc05271275e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6809","nonce":740986778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6811","nonce":740986778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6815","nonce":740986778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":740986778},{"type":"v1","addr":"172.21.15.96:6813","nonce":740986778}]},"public_addr":"172.21.15.96:6809/740986778","cluster_addr":"172.21.15.96:6811/740986778","heartbeat_back_addr":"172.21.15.96:6815/740986778","heartbeat_front_addr":"172.21.15.96:6813/740986778","state":["exists","up"]},{"osd":4,"uuid":"14ed2997-6237-4ca6-a17b-0c0ae87f0645","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6801","nonce":787691336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6803","nonce":787691336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6807","nonce":787691336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":787691336},{"type":"v1","addr":"172.21.15.150:6805","nonce":787691336}]},"public_addr":"172.21.15.150:6801/787691336","cluster_addr":"172.21.15.150:6803/787691336","heartbeat_back_addr":"172.21.15.150:6807/787691336","heartbeat_front_addr":"172.21.15.150:6805/787691336","state":["exists","up"]},{"osd":5,"uuid":"6ecd111f-9cc2-45a6-92f3-67f73bacfdad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6825","nonce":721780330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6827","nonce":721780330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6831","nonce":721780330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":721780330},{"type":"v1","addr":"172.21.15.150:6829","nonce":721780330}]},"public_addr":"172.21.15.150:6825/721780330","cluster_addr":"172.21.15.150:6827/721780330","heartbeat_back_addr":"172.21.15.150:6831/721780330","heartbeat_front_addr":"172.21.15.150:6829/721780330","state":["exists","up"]},{"osd":6,"uuid":"5032e618-100c-4318-9825-3e90a3c4b25e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6817","nonce":2103163729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6819","nonce":2103163729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6823","nonce":2103163729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2103163729},{"type":"v1","addr":"172.21.15.150:6821","nonce":2103163729}]},"public_addr":"172.21.15.150:6817/2103163729","cluster_addr":"172.21.15.150:6819/2103163729","heartbeat_back_addr":"172.21.15.150:6823/2103163729","heartbeat_front_addr":"172.21.15.150:6821/2103163729","state":["exists","up"]},{"osd":7,"uuid":"020cdc04-e2da-4af5-b6d9-674e88425186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6809","nonce":1517191652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6811","nonce":1517191652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6815","nonce":1517191652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":1517191652},{"type":"v1","addr":"172.21.15.150:6813","nonce":1517191652}]},"public_addr":"172.21.15.150:6809/1517191652","cluster_addr":"172.21.15.150:6811/1517191652","heartbeat_back_addr":"172.21.15.150:6815/1517191652","heartbeat_front_addr":"172.21.15.150:6813/1517191652","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.358516+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.767062+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.745538+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.615624+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.063897+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.205999+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:44.352769+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-22T01:13:43.943009+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1","osds":[4,7]},{"pgid":"3.16","osds":[2,0]}],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-22T01:17:30.174 INFO:tasks.ceph:Scrubbing osd.0 2024-09-22T01:17:30.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:30.333 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:30.333 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:30.333 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:30.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-22T01:17:30.639 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 0 to deep-scrub 2024-09-22T01:17:30.658 INFO:tasks.ceph:Scrubbing osd.1 2024-09-22T01:17:30.658 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:30.809 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:30.809 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:30.809 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:30.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-22T01:17:31.108 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 1 to deep-scrub 2024-09-22T01:17:31.127 INFO:tasks.ceph:Scrubbing osd.2 2024-09-22T01:17:31.127 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:31.273 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:31.273 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:31.273 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:31.289 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-22T01:17:31.572 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 2 to deep-scrub 2024-09-22T01:17:31.592 INFO:tasks.ceph:Scrubbing osd.3 2024-09-22T01:17:31.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:31.737 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:31.737 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:31.737 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:31.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-22T01:17:32.026 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 3 to deep-scrub 2024-09-22T01:17:32.044 INFO:tasks.ceph:Scrubbing osd.4 2024-09-22T01:17:32.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:32.199 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:32.200 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:32.200 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:32.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-22T01:17:32.502 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 4 to deep-scrub 2024-09-22T01:17:32.519 INFO:tasks.ceph:Scrubbing osd.5 2024-09-22T01:17:32.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:32.662 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:32.662 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:32.662 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:32.679 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-22T01:17:32.971 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 5 to deep-scrub 2024-09-22T01:17:32.990 INFO:tasks.ceph:Scrubbing osd.6 2024-09-22T01:17:32.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:33.138 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:33.138 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:33.138 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:33.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-22T01:17:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 6 to deep-scrub 2024-09-22T01:17:33.448 INFO:tasks.ceph:Scrubbing osd.7 2024-09-22T01:17:33.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-22T01:17:33.586 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-22T01:17:33.586 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-22T01:17:33.586 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-22T01:17:33.601 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-22T01:17:33.874 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 7 to deep-scrub 2024-09-22T01:17:33.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:34.175 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:34.192 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-22T01:17:33.588364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22012,"ondisk_log_size":22012,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":402292,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":382579,"kb_avail":754572428,"statfs":{"total":773094113280,"available":772682166272,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":391761456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004221"},"pg_stats":[{"pgid":"3.1f","version":"172'341","reported_seq":3321,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169634+0000","last_change":"2024-09-22T01:16:52.751303+0000","last_active":"2024-09-22T01:17:20.169634+0000","last_peered":"2024-09-22T01:17:20.169634+0000","last_clean":"2024-09-22T01:17:20.169634+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:20.169634+0000","last_undegraded":"2024-09-22T01:17:20.169634+0000","last_fullsized":"2024-09-22T01:17:20.169634+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:48:37.385919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"171'546","reported_seq":12830,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169570+0000","last_change":"2024-09-22T01:16:52.749692+0000","last_active":"2024-09-22T01:17:20.169570+0000","last_peered":"2024-09-22T01:17:20.169570+0000","last_clean":"2024-09-22T01:17:20.169570+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:20.169570+0000","last_undegraded":"2024-09-22T01:17:20.169570+0000","last_fullsized":"2024-09-22T01:17:20.169570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:25.809413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"171'478","reported_seq":20552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559885+0000","last_change":"2024-09-22T01:16:52.797433+0000","last_active":"2024-09-22T01:17:19.559885+0000","last_peered":"2024-09-22T01:17:19.559885+0000","last_clean":"2024-09-22T01:17:19.559885+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:19.559885+0000","last_undegraded":"2024-09-22T01:17:19.559885+0000","last_fullsized":"2024-09-22T01:17:19.559885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:28.321822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"172'693","reported_seq":13035,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168447+0000","last_change":"2024-09-22T01:16:52.667938+0000","last_active":"2024-09-22T01:17:20.168447+0000","last_peered":"2024-09-22T01:17:20.168447+0000","last_clean":"2024-09-22T01:17:20.168447+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:20.168447+0000","last_undegraded":"2024-09-22T01:17:20.168447+0000","last_fullsized":"2024-09-22T01:17:20.168447+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:15:09.326699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"169'420","reported_seq":5869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561110+0000","last_change":"2024-09-22T01:17:09.813022+0000","last_active":"2024-09-22T01:17:19.561110+0000","last_peered":"2024-09-22T01:17:19.561110+0000","last_clean":"2024-09-22T01:17:19.561110+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:19.561110+0000","last_undegraded":"2024-09-22T01:17:19.561110+0000","last_fullsized":"2024-09-22T01:17:19.561110+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:05.595060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"169'1187","reported_seq":17421,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168499+0000","last_change":"2024-09-22T01:16:52.667589+0000","last_active":"2024-09-22T01:17:20.168499+0000","last_peered":"2024-09-22T01:17:20.168499+0000","last_clean":"2024-09-22T01:17:20.168499+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:20.168499+0000","last_undegraded":"2024-09-22T01:17:20.168499+0000","last_fullsized":"2024-09-22T01:17:20.168499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:37:12.733173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"172'506","reported_seq":6612,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560970+0000","last_change":"2024-09-22T01:16:52.665385+0000","last_active":"2024-09-22T01:17:19.560970+0000","last_peered":"2024-09-22T01:17:19.560970+0000","last_clean":"2024-09-22T01:17:19.560970+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:19.560970+0000","last_undegraded":"2024-09-22T01:17:19.560970+0000","last_fullsized":"2024-09-22T01:17:19.560970+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:15:59.458846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"171'399","reported_seq":6646,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169725+0000","last_change":"2024-09-22T01:16:52.751468+0000","last_active":"2024-09-22T01:17:20.169725+0000","last_peered":"2024-09-22T01:17:20.169725+0000","last_clean":"2024-09-22T01:17:20.169725+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:20.169725+0000","last_undegraded":"2024-09-22T01:17:20.169725+0000","last_fullsized":"2024-09-22T01:17:20.169725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:08.584204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"170'571","reported_seq":5802,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169776+0000","last_change":"2024-09-22T01:16:52.752542+0000","last_active":"2024-09-22T01:17:20.169776+0000","last_peered":"2024-09-22T01:17:20.169776+0000","last_clean":"2024-09-22T01:17:20.169776+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:20.169776+0000","last_undegraded":"2024-09-22T01:17:20.169776+0000","last_fullsized":"2024-09-22T01:17:20.169776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:29:35.155600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"170'528","reported_seq":13509,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783820+0000","last_change":"2024-09-22T01:16:52.657904+0000","last_active":"2024-09-22T01:17:19.783820+0000","last_peered":"2024-09-22T01:17:19.783820+0000","last_clean":"2024-09-22T01:17:19.783820+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:19.783820+0000","last_undegraded":"2024-09-22T01:17:19.783820+0000","last_fullsized":"2024-09-22T01:17:19.783820+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:08:41.269146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"195'863","reported_seq":4134,"reported_epoch":195,"state":"active+clean+scrubbing+deep","last_fresh":"2024-09-22T01:17:32.590888+0000","last_change":"2024-09-22T01:17:32.578122+0000","last_active":"2024-09-22T01:17:32.590888+0000","last_peered":"2024-09-22T01:17:32.590888+0000","last_clean":"2024-09-22T01:17:32.590888+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:32.590888+0000","last_undegraded":"2024-09-22T01:17:32.590888+0000","last_fullsized":"2024-09-22T01:17:32.590888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":12,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'755","reported_seq":10661,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559715+0000","last_change":"2024-09-22T01:16:52.797032+0000","last_active":"2024-09-22T01:17:19.559715+0000","last_peered":"2024-09-22T01:17:19.559715+0000","last_clean":"2024-09-22T01:17:19.559715+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:19.559715+0000","last_undegraded":"2024-09-22T01:17:19.559715+0000","last_fullsized":"2024-09-22T01:17:19.559715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:20:25.188258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560176+0000","last_change":"2024-09-22T01:14:10.540508+0000","last_active":"2024-09-22T01:17:19.560176+0000","last_peered":"2024-09-22T01:17:19.560176+0000","last_clean":"2024-09-22T01:17:19.560176+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:19.560176+0000","last_undegraded":"2024-09-22T01:17:19.560176+0000","last_fullsized":"2024-09-22T01:17:19.560176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:30:05.408412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"172'656","reported_seq":4959,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.572157+0000","last_change":"2024-09-22T01:17:19.572157+0000","last_active":"2024-09-22T01:17:19.572157+0000","last_peered":"2024-09-22T01:17:19.572157+0000","last_clean":"2024-09-22T01:17:19.572157+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:19.572157+0000","last_undegraded":"2024-09-22T01:17:19.572157+0000","last_fullsized":"2024-09-22T01:17:19.572157+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:02:12.989959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":192,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561340+0000","last_change":"2024-09-22T01:15:59.607487+0000","last_active":"2024-09-22T01:17:19.561340+0000","last_peered":"2024-09-22T01:17:19.561340+0000","last_clean":"2024-09-22T01:17:19.561340+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:19.561340+0000","last_undegraded":"2024-09-22T01:17:19.561340+0000","last_fullsized":"2024-09-22T01:17:19.561340+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:45:26.466106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"165'378","reported_seq":4323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561351+0000","last_change":"2024-09-22T01:16:54.544575+0000","last_active":"2024-09-22T01:17:19.561351+0000","last_peered":"2024-09-22T01:17:19.561351+0000","last_clean":"2024-09-22T01:17:19.561351+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:19.561351+0000","last_undegraded":"2024-09-22T01:17:19.561351+0000","last_fullsized":"2024-09-22T01:17:19.561351+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:53.597290+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560076+0000","last_change":"2024-09-22T01:14:10.536906+0000","last_active":"2024-09-22T01:17:19.560076+0000","last_peered":"2024-09-22T01:17:19.560076+0000","last_clean":"2024-09-22T01:17:19.560076+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:19.560076+0000","last_undegraded":"2024-09-22T01:17:19.560076+0000","last_fullsized":"2024-09-22T01:17:19.560076+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:54.993525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"171'663","reported_seq":3694,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558957+0000","last_change":"2024-09-22T01:17:13.855610+0000","last_active":"2024-09-22T01:17:19.558957+0000","last_peered":"2024-09-22T01:17:19.558957+0000","last_clean":"2024-09-22T01:17:19.558957+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:19.558957+0000","last_undegraded":"2024-09-22T01:17:19.558957+0000","last_fullsized":"2024-09-22T01:17:19.558957+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:41:37.079389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":360,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169674+0000","last_change":"2024-09-22T01:13:57.040204+0000","last_active":"2024-09-22T01:17:20.169674+0000","last_peered":"2024-09-22T01:17:20.169674+0000","last_clean":"2024-09-22T01:17:20.169674+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:20.169674+0000","last_undegraded":"2024-09-22T01:17:20.169674+0000","last_fullsized":"2024-09-22T01:17:20.169674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:56.843683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"169'1157","reported_seq":8761,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783798+0000","last_change":"2024-09-22T01:16:52.659217+0000","last_active":"2024-09-22T01:17:19.783798+0000","last_peered":"2024-09-22T01:17:19.783798+0000","last_clean":"2024-09-22T01:17:19.783798+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:19.783798+0000","last_undegraded":"2024-09-22T01:17:19.783798+0000","last_fullsized":"2024-09-22T01:17:19.783798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1157,"log_dups_size":0,"ondisk_log_size":1157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:24:59.586757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"172'965","reported_seq":6983,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169020+0000","last_change":"2024-09-22T01:16:52.673965+0000","last_active":"2024-09-22T01:17:20.169020+0000","last_peered":"2024-09-22T01:17:20.169020+0000","last_clean":"2024-09-22T01:17:20.169020+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:20.169020+0000","last_undegraded":"2024-09-22T01:17:20.169020+0000","last_fullsized":"2024-09-22T01:17:20.169020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":345,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559863+0000","last_change":"2024-09-22T01:14:10.546309+0000","last_active":"2024-09-22T01:17:19.559863+0000","last_peered":"2024-09-22T01:17:19.559863+0000","last_clean":"2024-09-22T01:17:19.559863+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:19.559863+0000","last_undegraded":"2024-09-22T01:17:19.559863+0000","last_fullsized":"2024-09-22T01:17:19.559863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:48:36.900659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"171'515","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.169382+0000","last_change":"2024-09-22T01:17:18.573188+0000","last_active":"2024-09-22T01:17:20.169382+0000","last_peered":"2024-09-22T01:17:20.169382+0000","last_clean":"2024-09-22T01:17:20.169382+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:20.169382+0000","last_undegraded":"2024-09-22T01:17:20.169382+0000","last_fullsized":"2024-09-22T01:17:20.169382+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:06:22.076986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.562928+0000","last_change":"2024-09-22T01:17:14.555013+0000","last_active":"2024-09-22T01:17:19.562928+0000","last_peered":"2024-09-22T01:17:19.562928+0000","last_clean":"2024-09-22T01:17:19.562928+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:19.562928+0000","last_undegraded":"2024-09-22T01:17:19.562928+0000","last_fullsized":"2024-09-22T01:17:19.562928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:49:09.221692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"171'576","reported_seq":7036,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560957+0000","last_change":"2024-09-22T01:17:09.814036+0000","last_active":"2024-09-22T01:17:19.560957+0000","last_peered":"2024-09-22T01:17:19.560957+0000","last_clean":"2024-09-22T01:17:19.560957+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:19.560957+0000","last_undegraded":"2024-09-22T01:17:19.560957+0000","last_fullsized":"2024-09-22T01:17:19.560957+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:06:21.006114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559448+0000","last_change":"2024-09-22T01:14:10.536517+0000","last_active":"2024-09-22T01:17:19.559448+0000","last_peered":"2024-09-22T01:17:19.559448+0000","last_clean":"2024-09-22T01:17:19.559448+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:19.559448+0000","last_undegraded":"2024-09-22T01:17:19.559448+0000","last_fullsized":"2024-09-22T01:17:19.559448+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T13:02:31.679918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'673","reported_seq":6578,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.561235+0000","last_change":"2024-09-22T01:16:52.662219+0000","last_active":"2024-09-22T01:17:19.561235+0000","last_peered":"2024-09-22T01:17:19.561235+0000","last_clean":"2024-09-22T01:17:19.561235+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:19.561235+0000","last_undegraded":"2024-09-22T01:17:19.561235+0000","last_fullsized":"2024-09-22T01:17:19.561235+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:34:19.252628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":363,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559762+0000","last_change":"2024-09-22T01:14:38.760875+0000","last_active":"2024-09-22T01:17:19.559762+0000","last_peered":"2024-09-22T01:17:19.559762+0000","last_clean":"2024-09-22T01:17:19.559762+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:19.559762+0000","last_undegraded":"2024-09-22T01:17:19.559762+0000","last_fullsized":"2024-09-22T01:17:19.559762+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:53.985269+0000","last_clean_scrub_stamp":"2024-09-22T01:13:53.985269+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:44:49.075193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":513,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168467+0000","last_change":"2024-09-22T01:13:51.179388+0000","last_active":"2024-09-22T01:17:20.168467+0000","last_peered":"2024-09-22T01:17:20.168467+0000","last_clean":"2024-09-22T01:17:20.168467+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:20.168467+0000","last_undegraded":"2024-09-22T01:17:20.168467+0000","last_fullsized":"2024-09-22T01:17:20.168467+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:13:49.947856+0000","last_clean_scrub_stamp":"2024-09-22T01:13:49.947856+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:14:24.555532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"170'513","reported_seq":4307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.560902+0000","last_change":"2024-09-22T01:17:18.574299+0000","last_active":"2024-09-22T01:17:19.560902+0000","last_peered":"2024-09-22T01:17:19.560902+0000","last_clean":"2024-09-22T01:17:19.560902+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:19.560902+0000","last_undegraded":"2024-09-22T01:17:19.560902+0000","last_fullsized":"2024-09-22T01:17:19.560902+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:31:45.705150+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"171'1003","reported_seq":19415,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559430+0000","last_change":"2024-09-22T01:16:52.799459+0000","last_active":"2024-09-22T01:17:19.559430+0000","last_peered":"2024-09-22T01:17:19.559430+0000","last_clean":"2024-09-22T01:17:19.559430+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:19.559430+0000","last_undegraded":"2024-09-22T01:17:19.559430+0000","last_fullsized":"2024-09-22T01:17:19.559430+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:24:35.438063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"171'712","reported_seq":4584,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559616+0000","last_change":"2024-09-22T01:16:52.800145+0000","last_active":"2024-09-22T01:17:19.559616+0000","last_peered":"2024-09-22T01:17:19.559616+0000","last_clean":"2024-09-22T01:17:19.559616+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:19.559616+0000","last_undegraded":"2024-09-22T01:17:19.559616+0000","last_fullsized":"2024-09-22T01:17:19.559616+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:37.003710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"171'575","reported_seq":4970,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.570549+0000","last_change":"2024-09-22T01:17:19.570549+0000","last_active":"2024-09-22T01:17:19.570549+0000","last_peered":"2024-09-22T01:17:19.570549+0000","last_clean":"2024-09-22T01:17:19.570549+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:19.570549+0000","last_undegraded":"2024-09-22T01:17:19.570549+0000","last_fullsized":"2024-09-22T01:17:19.570549+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:27.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"171'707","reported_seq":6327,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168522+0000","last_change":"2024-09-22T01:17:18.572800+0000","last_active":"2024-09-22T01:17:20.168522+0000","last_peered":"2024-09-22T01:17:20.168522+0000","last_clean":"2024-09-22T01:17:20.168522+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:20.168522+0000","last_undegraded":"2024-09-22T01:17:20.168522+0000","last_fullsized":"2024-09-22T01:17:20.168522+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"169'687","reported_seq":3051,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559985+0000","last_change":"2024-09-22T01:16:52.798229+0000","last_active":"2024-09-22T01:17:19.559985+0000","last_peered":"2024-09-22T01:17:19.559985+0000","last_clean":"2024-09-22T01:17:19.559985+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:19.559985+0000","last_undegraded":"2024-09-22T01:17:19.559985+0000","last_fullsized":"2024-09-22T01:17:19.559985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:47:23.850481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"172'896","reported_seq":4151,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.783937+0000","last_change":"2024-09-22T01:16:52.658756+0000","last_active":"2024-09-22T01:17:19.783937+0000","last_peered":"2024-09-22T01:17:19.783937+0000","last_clean":"2024-09-22T01:17:19.783937+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:19.783937+0000","last_undegraded":"2024-09-22T01:17:19.783937+0000","last_fullsized":"2024-09-22T01:17:19.783937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:43:29.707003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"195'729","reported_seq":4128,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:31.609032+0000","last_change":"2024-09-22T01:17:31.609088+0000","last_active":"2024-09-22T01:17:31.609032+0000","last_peered":"2024-09-22T01:17:31.609032+0000","last_clean":"2024-09-22T01:17:31.609032+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:31.609032+0000","last_undegraded":"2024-09-22T01:17:31.609032+0000","last_fullsized":"2024-09-22T01:17:31.609032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'729","last_scrub_stamp":"2024-09-22T01:17:31.608964+0000","last_deep_scrub":"195'729","last_deep_scrub_stamp":"2024-09-22T01:17:31.608964+0000","last_clean_scrub_stamp":"2024-09-22T01:17:31.608964+0000","objects_scrubbed":16,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:57:54.509493+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"164'661","reported_seq":3855,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:20.168348+0000","last_change":"2024-09-22T01:16:52.667303+0000","last_active":"2024-09-22T01:17:20.168348+0000","last_peered":"2024-09-22T01:17:20.168348+0000","last_clean":"2024-09-22T01:17:20.168348+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:20.168348+0000","last_undegraded":"2024-09-22T01:17:20.168348+0000","last_fullsized":"2024-09-22T01:17:20.168348+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:54:03.744675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"170'950","reported_seq":5320,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559456+0000","last_change":"2024-09-22T01:16:52.799144+0000","last_active":"2024-09-22T01:17:19.559456+0000","last_peered":"2024-09-22T01:17:19.559456+0000","last_clean":"2024-09-22T01:17:19.559456+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:19.559456+0000","last_undegraded":"2024-09-22T01:17:19.559456+0000","last_fullsized":"2024-09-22T01:17:19.559456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:39:55.050827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"167'809","reported_seq":5318,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.558881+0000","last_change":"2024-09-22T01:17:13.857704+0000","last_active":"2024-09-22T01:17:19.558881+0000","last_peered":"2024-09-22T01:17:19.558881+0000","last_clean":"2024-09-22T01:17:19.558881+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:19.558881+0000","last_undegraded":"2024-09-22T01:17:19.558881+0000","last_fullsized":"2024-09-22T01:17:19.558881+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:33.241916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"171'780","reported_seq":6046,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:19.559659+0000","last_change":"2024-09-22T01:16:52.798389+0000","last_active":"2024-09-22T01:17:19.559659+0000","last_peered":"2024-09-22T01:17:19.559659+0000","last_clean":"2024-09-22T01:17:19.559659+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:19.559659+0000","last_undegraded":"2024-09-22T01:17:19.559659+0000","last_fullsized":"2024-09-22T01:17:19.559659+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:14:04.077303+0000","last_clean_scrub_stamp":"2024-09-22T01:14:04.077303+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:01:04.024641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21892,"ondisk_log_size":21892,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29328,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342512,"statfs":{"total":96636764160,"available":96606732288,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51532,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320308,"statfs":{"total":96636764160,"available":96583995392,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52112,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319728,"statfs":{"total":96636764160,"available":96583401472,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59852,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311988,"statfs":{"total":96636764160,"available":96575475712,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509490,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56280,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315560,"statfs":{"total":96636764160,"available":96579133440,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:34.196 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.196 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.196 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.197 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.198 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.199 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-22T01:13:53.985269+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=53, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-22T01:13:49.947856+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=13, tm_sec=49, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.200 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.201 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.201 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.201 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.201 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-22T01:14:04.077303+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=14, tm_sec=4, tm_wday=6, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=22, tm_hour=1, tm_min=17, tm_sec=29, tm_wday=6, tm_yday=266, tm_isdst=0) 2024-09-22T01:17:34.201 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-22T01:17:54.202 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-22T01:17:54.519 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-22T01:17:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-22T01:17:54.538 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-22T01:17:53.594449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":537,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":537,"num_whiteouts":529,"num_read":271306,"num_read_kb":970842,"num_write":23941,"num_write_kb":379989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1979,"num_bytes_recovered":92954314,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22521,"ondisk_log_size":22521,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406392,"kb_used_data":19512,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754568328,"statfs":{"total":773094113280,"available":772677967872,"internally_reserved":0,"allocated":19980288,"data_stored":10225740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":396021296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"3.1f","version":"195'349","reported_seq":3338,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:35.020729+0000","last_change":"2024-09-22T01:17:35.020863+0000","last_active":"2024-09-22T01:17:35.020729+0000","last_peered":"2024-09-22T01:17:35.020729+0000","last_clean":"2024-09-22T01:17:35.020729+0000","last_became_active":"2024-09-22T01:14:04.517248+0000","last_became_peered":"2024-09-22T01:14:04.517248+0000","last_unstale":"2024-09-22T01:17:35.020729+0000","last_undegraded":"2024-09-22T01:17:35.020729+0000","last_fullsized":"2024-09-22T01:17:35.020729+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'349","last_scrub_stamp":"2024-09-22T01:17:35.020499+0000","last_deep_scrub":"195'349","last_deep_scrub_stamp":"2024-09-22T01:17:35.020499+0000","last_clean_scrub_stamp":"2024-09-22T01:17:35.020499+0000","objects_scrubbed":8,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:04.816412+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.003067335,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2479,"num_read_kb":21993,"num_write":296,"num_write_kb":6411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"195'559","reported_seq":12852,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:38.064743+0000","last_change":"2024-09-22T01:17:38.064846+0000","last_active":"2024-09-22T01:17:38.064743+0000","last_peered":"2024-09-22T01:17:38.064743+0000","last_clean":"2024-09-22T01:17:38.064743+0000","last_became_active":"2024-09-22T01:14:38.547395+0000","last_became_peered":"2024-09-22T01:14:38.547395+0000","last_unstale":"2024-09-22T01:17:38.064743+0000","last_undegraded":"2024-09-22T01:17:38.064743+0000","last_fullsized":"2024-09-22T01:17:38.064743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'559","last_scrub_stamp":"2024-09-22T01:17:38.064540+0000","last_deep_scrub":"195'559","last_deep_scrub_stamp":"2024-09-22T01:17:38.064540+0000","last_clean_scrub_stamp":"2024-09-22T01:17:38.064540+0000","objects_scrubbed":13,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:53:32.014142+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0014768299999999999,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":12573,"num_read_kb":37134,"num_write":543,"num_write_kb":16355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":11970160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"195'492","reported_seq":20575,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:39.331569+0000","last_change":"2024-09-22T01:17:39.331669+0000","last_active":"2024-09-22T01:17:39.331569+0000","last_peered":"2024-09-22T01:17:39.331569+0000","last_clean":"2024-09-22T01:17:39.331569+0000","last_became_active":"2024-09-22T01:14:10.539215+0000","last_became_peered":"2024-09-22T01:14:10.539215+0000","last_unstale":"2024-09-22T01:17:39.331569+0000","last_undegraded":"2024-09-22T01:17:39.331569+0000","last_fullsized":"2024-09-22T01:17:39.331569+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'492","last_scrub_stamp":"2024-09-22T01:17:39.331372+0000","last_deep_scrub":"195'492","last_deep_scrub_stamp":"2024-09-22T01:17:39.331372+0000","last_clean_scrub_stamp":"2024-09-22T01:17:39.331372+0000","objects_scrubbed":14,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:17:38.481370+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.00124098,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":21023,"num_read_kb":28384,"num_write":420,"num_write_kb":10288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"195'711","reported_seq":13062,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:32.166846+0000","last_change":"2024-09-22T01:17:32.166939+0000","last_active":"2024-09-22T01:17:32.166846+0000","last_peered":"2024-09-22T01:17:32.166846+0000","last_clean":"2024-09-22T01:17:32.166846+0000","last_became_active":"2024-09-22T01:14:04.514710+0000","last_became_peered":"2024-09-22T01:14:04.514710+0000","last_unstale":"2024-09-22T01:17:32.166846+0000","last_undegraded":"2024-09-22T01:17:32.166846+0000","last_fullsized":"2024-09-22T01:17:32.166846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'711","last_scrub_stamp":"2024-09-22T01:17:32.166682+0000","last_deep_scrub":"195'711","last_deep_scrub_stamp":"2024-09-22T01:17:32.166682+0000","last_clean_scrub_stamp":"2024-09-22T01:17:32.166682+0000","objects_scrubbed":18,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:19.380936+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0015161599999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":17,"num_read":13349,"num_read_kb":39073,"num_write":796,"num_write_kb":12892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"195'438","reported_seq":5896,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:36.747306+0000","last_change":"2024-09-22T01:17:36.747473+0000","last_active":"2024-09-22T01:17:36.747306+0000","last_peered":"2024-09-22T01:17:36.747306+0000","last_clean":"2024-09-22T01:17:36.747306+0000","last_became_active":"2024-09-22T01:17:09.810119+0000","last_became_peered":"2024-09-22T01:17:09.810119+0000","last_unstale":"2024-09-22T01:17:36.747306+0000","last_undegraded":"2024-09-22T01:17:36.747306+0000","last_fullsized":"2024-09-22T01:17:36.747306+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'438","last_scrub_stamp":"2024-09-22T01:17:36.747022+0000","last_deep_scrub":"195'438","last_deep_scrub_stamp":"2024-09-22T01:17:36.747022+0000","last_clean_scrub_stamp":"2024-09-22T01:17:36.747022+0000","objects_scrubbed":18,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:16:15.255085+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.0053275759999999997,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":5344,"num_read_kb":11450,"num_write":326,"num_write_kb":4895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"195'1197","reported_seq":17440,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.206996+0000","last_change":"2024-09-22T01:17:34.207094+0000","last_active":"2024-09-22T01:17:34.206996+0000","last_peered":"2024-09-22T01:17:34.206996+0000","last_clean":"2024-09-22T01:17:34.206996+0000","last_became_active":"2024-09-22T01:14:04.516305+0000","last_became_peered":"2024-09-22T01:14:04.516305+0000","last_unstale":"2024-09-22T01:17:34.206996+0000","last_undegraded":"2024-09-22T01:17:34.206996+0000","last_fullsized":"2024-09-22T01:17:34.206996+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1197","last_scrub_stamp":"2024-09-22T01:17:34.206775+0000","last_deep_scrub":"195'1197","last_deep_scrub_stamp":"2024-09-22T01:17:34.206775+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.206775+0000","objects_scrubbed":10,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:20:15.686106+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.00116318,"stat_sum":{"num_bytes":19,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":20193,"num_read_kb":77953,"num_write":1714,"num_write_kb":27243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"195'521","reported_seq":6636,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.764994+0000","last_change":"2024-09-22T01:17:33.765113+0000","last_active":"2024-09-22T01:17:33.764994+0000","last_peered":"2024-09-22T01:17:33.764994+0000","last_clean":"2024-09-22T01:17:33.764994+0000","last_became_active":"2024-09-22T01:15:59.609348+0000","last_became_peered":"2024-09-22T01:15:59.609348+0000","last_unstale":"2024-09-22T01:17:33.764994+0000","last_undegraded":"2024-09-22T01:17:33.764994+0000","last_fullsized":"2024-09-22T01:17:33.764994+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'521","last_scrub_stamp":"2024-09-22T01:17:33.764783+0000","last_deep_scrub":"195'521","last_deep_scrub_stamp":"2024-09-22T01:17:33.764783+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.764783+0000","objects_scrubbed":15,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:03:03.216889+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0037505020000000002,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7396,"num_read_kb":28401,"num_write":383,"num_write_kb":6656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"195'421","reported_seq":6677,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:36.077745+0000","last_change":"2024-09-22T01:17:36.077854+0000","last_active":"2024-09-22T01:17:36.077745+0000","last_peered":"2024-09-22T01:17:36.077745+0000","last_clean":"2024-09-22T01:17:36.077745+0000","last_became_active":"2024-09-22T01:14:04.529798+0000","last_became_peered":"2024-09-22T01:14:04.529798+0000","last_unstale":"2024-09-22T01:17:36.077745+0000","last_undegraded":"2024-09-22T01:17:36.077745+0000","last_fullsized":"2024-09-22T01:17:36.077745+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'421","last_scrub_stamp":"2024-09-22T01:17:36.077559+0000","last_deep_scrub":"195'421","last_deep_scrub_stamp":"2024-09-22T01:17:36.077559+0000","last_clean_scrub_stamp":"2024-09-22T01:17:36.077559+0000","objects_scrubbed":22,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:55:01.013393+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0029262120000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6051,"num_read_kb":39372,"num_write":348,"num_write_kb":21846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"195'584","reported_seq":5824,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:37.035572+0000","last_change":"2024-09-22T01:17:37.035734+0000","last_active":"2024-09-22T01:17:37.035572+0000","last_peered":"2024-09-22T01:17:37.035572+0000","last_clean":"2024-09-22T01:17:37.035572+0000","last_became_active":"2024-09-22T01:14:10.539032+0000","last_became_peered":"2024-09-22T01:14:10.539032+0000","last_unstale":"2024-09-22T01:17:37.035572+0000","last_undegraded":"2024-09-22T01:17:37.035572+0000","last_fullsized":"2024-09-22T01:17:37.035572+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'584","last_scrub_stamp":"2024-09-22T01:17:37.035337+0000","last_deep_scrub":"195'584","last_deep_scrub_stamp":"2024-09-22T01:17:37.035337+0000","last_clean_scrub_stamp":"2024-09-22T01:17:37.035337+0000","objects_scrubbed":13,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:15:10.400293+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0038837670000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6518,"num_read_kb":34466,"num_write":527,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"195'555","reported_seq":13546,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.718559+0000","last_change":"2024-09-22T01:17:33.718615+0000","last_active":"2024-09-22T01:17:33.718559+0000","last_peered":"2024-09-22T01:17:33.718559+0000","last_clean":"2024-09-22T01:17:33.718559+0000","last_became_active":"2024-09-22T01:16:09.528900+0000","last_became_peered":"2024-09-22T01:16:09.528900+0000","last_unstale":"2024-09-22T01:17:33.718559+0000","last_undegraded":"2024-09-22T01:17:33.718559+0000","last_fullsized":"2024-09-22T01:17:33.718559+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'555","last_scrub_stamp":"2024-09-22T01:17:33.718471+0000","last_deep_scrub":"195'555","last_deep_scrub_stamp":"2024-09-22T01:17:33.718471+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.718471+0000","objects_scrubbed":27,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T11:23:51.759513+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.00119929,"stat_sum":{"num_bytes":0,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":27,"num_read":12286,"num_read_kb":27193,"num_write":505,"num_write_kb":13008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":12297333,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"195'863","reported_seq":4148,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:32.595655+0000","last_change":"2024-09-22T01:17:32.595731+0000","last_active":"2024-09-22T01:17:32.595655+0000","last_peered":"2024-09-22T01:17:32.595655+0000","last_clean":"2024-09-22T01:17:32.595655+0000","last_became_active":"2024-09-22T01:14:04.528535+0000","last_became_peered":"2024-09-22T01:14:04.528535+0000","last_unstale":"2024-09-22T01:17:32.595655+0000","last_undegraded":"2024-09-22T01:17:32.595655+0000","last_fullsized":"2024-09-22T01:17:32.595655+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'863","last_scrub_stamp":"2024-09-22T01:17:32.595526+0000","last_deep_scrub":"195'863","last_deep_scrub_stamp":"2024-09-22T01:17:32.595526+0000","last_clean_scrub_stamp":"2024-09-22T01:17:32.595526+0000","objects_scrubbed":12,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:59:41.312905+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0020912890000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5715,"num_read_kb":18819,"num_write":1189,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"195'774","reported_seq":10689,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:42.372129+0000","last_change":"2024-09-22T01:17:42.372243+0000","last_active":"2024-09-22T01:17:42.372129+0000","last_peered":"2024-09-22T01:17:42.372129+0000","last_clean":"2024-09-22T01:17:42.372129+0000","last_became_active":"2024-09-22T01:14:41.578118+0000","last_became_peered":"2024-09-22T01:14:41.578118+0000","last_unstale":"2024-09-22T01:17:42.372129+0000","last_undegraded":"2024-09-22T01:17:42.372129+0000","last_fullsized":"2024-09-22T01:17:42.372129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'774","last_scrub_stamp":"2024-09-22T01:17:42.371932+0000","last_deep_scrub":"195'774","last_deep_scrub_stamp":"2024-09-22T01:17:42.371932+0000","last_clean_scrub_stamp":"2024-09-22T01:17:42.371932+0000","objects_scrubbed":19,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:54.242312+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00089118300000000004,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":12362,"num_read_kb":35727,"num_write":918,"num_write_kb":12779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2904236,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":350,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:35.288253+0000","last_change":"2024-09-22T01:17:35.288304+0000","last_active":"2024-09-22T01:17:35.288253+0000","last_peered":"2024-09-22T01:17:35.288253+0000","last_clean":"2024-09-22T01:17:35.288253+0000","last_became_active":"2024-09-22T01:14:10.540213+0000","last_became_peered":"2024-09-22T01:14:10.540213+0000","last_unstale":"2024-09-22T01:17:35.288253+0000","last_undegraded":"2024-09-22T01:17:35.288253+0000","last_fullsized":"2024-09-22T01:17:35.288253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:35.288053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:35.288053+0000","last_clean_scrub_stamp":"2024-09-22T01:17:35.288053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:38.183482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"195'671","reported_seq":4983,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:39.696070+0000","last_change":"2024-09-22T01:17:39.696156+0000","last_active":"2024-09-22T01:17:39.696070+0000","last_peered":"2024-09-22T01:17:39.696070+0000","last_clean":"2024-09-22T01:17:39.696070+0000","last_became_active":"2024-09-22T01:17:19.568401+0000","last_became_peered":"2024-09-22T01:17:19.568401+0000","last_unstale":"2024-09-22T01:17:39.696070+0000","last_undegraded":"2024-09-22T01:17:39.696070+0000","last_fullsized":"2024-09-22T01:17:39.696070+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'671","last_scrub_stamp":"2024-09-22T01:17:39.695878+0000","last_deep_scrub":"195'671","last_deep_scrub_stamp":"2024-09-22T01:17:39.695878+0000","last_clean_scrub_stamp":"2024-09-22T01:17:39.695878+0000","objects_scrubbed":15,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T10:16:25.349385+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.002756807,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6737,"num_read_kb":41404,"num_write":556,"num_write_kb":23297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":217,"num_bytes_recovered":21084122,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":201,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:32.723700+0000","last_change":"2024-09-22T01:17:32.723756+0000","last_active":"2024-09-22T01:17:32.723700+0000","last_peered":"2024-09-22T01:17:32.723700+0000","last_clean":"2024-09-22T01:17:32.723700+0000","last_became_active":"2024-09-22T01:15:59.607287+0000","last_became_peered":"2024-09-22T01:15:59.607287+0000","last_unstale":"2024-09-22T01:17:32.723700+0000","last_undegraded":"2024-09-22T01:17:32.723700+0000","last_fullsized":"2024-09-22T01:17:32.723700+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:32.723500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:32.723500+0000","last_clean_scrub_stamp":"2024-09-22T01:17:32.723500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:07:47.117878+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"195'394","reported_seq":4348,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:35.766586+0000","last_change":"2024-09-22T01:17:35.766692+0000","last_active":"2024-09-22T01:17:35.766586+0000","last_peered":"2024-09-22T01:17:35.766586+0000","last_clean":"2024-09-22T01:17:35.766586+0000","last_became_active":"2024-09-22T01:16:54.543564+0000","last_became_peered":"2024-09-22T01:16:54.543564+0000","last_unstale":"2024-09-22T01:17:35.766586+0000","last_undegraded":"2024-09-22T01:17:35.766586+0000","last_fullsized":"2024-09-22T01:17:35.766586+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'394","last_scrub_stamp":"2024-09-22T01:17:35.766360+0000","last_deep_scrub":"195'394","last_deep_scrub_stamp":"2024-09-22T01:17:35.766360+0000","last_clean_scrub_stamp":"2024-09-22T01:17:35.766360+0000","objects_scrubbed":16,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:23:03.764509+0000","scrub_duration":27,"objects_trimmed":1,"snaptrim_duration":0.013967083999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4224,"num_read_kb":26520,"num_write":298,"num_write_kb":9857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":350,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.321839+0000","last_change":"2024-09-22T01:17:34.321889+0000","last_active":"2024-09-22T01:17:34.321839+0000","last_peered":"2024-09-22T01:17:34.321839+0000","last_clean":"2024-09-22T01:17:34.321839+0000","last_became_active":"2024-09-22T01:14:10.535712+0000","last_became_peered":"2024-09-22T01:14:10.535712+0000","last_unstale":"2024-09-22T01:17:34.321839+0000","last_undegraded":"2024-09-22T01:17:34.321839+0000","last_fullsized":"2024-09-22T01:17:34.321839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:34.321654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:34.321654+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.321654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T12:46:04.200644+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"195'683","reported_seq":3723,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.834704+0000","last_change":"2024-09-22T01:17:33.834821+0000","last_active":"2024-09-22T01:17:33.834704+0000","last_peered":"2024-09-22T01:17:33.834704+0000","last_clean":"2024-09-22T01:17:33.834704+0000","last_became_active":"2024-09-22T01:17:13.851461+0000","last_became_peered":"2024-09-22T01:17:13.851461+0000","last_unstale":"2024-09-22T01:17:33.834704+0000","last_undegraded":"2024-09-22T01:17:33.834704+0000","last_fullsized":"2024-09-22T01:17:33.834704+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'683","last_scrub_stamp":"2024-09-22T01:17:33.834499+0000","last_deep_scrub":"195'683","last_deep_scrub_stamp":"2024-09-22T01:17:33.834499+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.834499+0000","objects_scrubbed":20,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:16.190279+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.001815349,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5704,"num_read_kb":19663,"num_write":778,"num_write_kb":6459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.966442+0000","last_change":"2024-09-22T01:17:33.966497+0000","last_active":"2024-09-22T01:17:33.966442+0000","last_peered":"2024-09-22T01:17:33.966442+0000","last_clean":"2024-09-22T01:17:33.966442+0000","last_became_active":"2024-09-22T01:13:55.342648+0000","last_became_peered":"2024-09-22T01:13:55.342648+0000","last_unstale":"2024-09-22T01:17:33.966442+0000","last_undegraded":"2024-09-22T01:17:33.966442+0000","last_fullsized":"2024-09-22T01:17:33.966442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:33.966226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:33.966226+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.966226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:43.566287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089643299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"195'1178","reported_seq":8791,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:31.728202+0000","last_change":"2024-09-22T01:17:31.728264+0000","last_active":"2024-09-22T01:17:31.728202+0000","last_peered":"2024-09-22T01:17:31.728202+0000","last_clean":"2024-09-22T01:17:31.728202+0000","last_became_active":"2024-09-22T01:14:41.569840+0000","last_became_peered":"2024-09-22T01:14:41.569840+0000","last_unstale":"2024-09-22T01:17:31.728202+0000","last_undegraded":"2024-09-22T01:17:31.728202+0000","last_fullsized":"2024-09-22T01:17:31.728202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1178","last_scrub_stamp":"2024-09-22T01:17:31.728109+0000","last_deep_scrub":"195'1178","last_deep_scrub_stamp":"2024-09-22T01:17:31.728109+0000","last_clean_scrub_stamp":"2024-09-22T01:17:31.728109+0000","objects_scrubbed":21,"log_size":1178,"log_dups_size":0,"ondisk_log_size":1178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:43:58.451509+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.002034962,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11582,"num_read_kb":39744,"num_write":1720,"num_write_kb":16626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":404012,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"195'982","reported_seq":7008,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.615635+0000","last_change":"2024-09-22T01:17:33.615689+0000","last_active":"2024-09-22T01:17:33.615635+0000","last_peered":"2024-09-22T01:17:33.615635+0000","last_clean":"2024-09-22T01:17:33.615635+0000","last_became_active":"2024-09-22T01:14:38.547713+0000","last_became_peered":"2024-09-22T01:14:38.547713+0000","last_unstale":"2024-09-22T01:17:33.615635+0000","last_undegraded":"2024-09-22T01:17:33.615635+0000","last_fullsized":"2024-09-22T01:17:33.615635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'982","last_scrub_stamp":"2024-09-22T01:17:33.615554+0000","last_deep_scrub":"195'982","last_deep_scrub_stamp":"2024-09-22T01:17:33.615554+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.615554+0000","objects_scrubbed":17,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:32:54.156944+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0023487099999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":7485,"num_read_kb":29853,"num_write":1001,"num_write_kb":11436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4073802,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"195'3","reported_seq":355,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:38.280861+0000","last_change":"2024-09-22T01:17:38.280922+0000","last_active":"2024-09-22T01:17:38.280861+0000","last_peered":"2024-09-22T01:17:38.280861+0000","last_clean":"2024-09-22T01:17:38.280861+0000","last_became_active":"2024-09-22T01:14:10.536674+0000","last_became_peered":"2024-09-22T01:14:10.536674+0000","last_unstale":"2024-09-22T01:17:38.280861+0000","last_undegraded":"2024-09-22T01:17:38.280861+0000","last_fullsized":"2024-09-22T01:17:38.280861+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'3","last_scrub_stamp":"2024-09-22T01:17:38.280677+0000","last_deep_scrub":"195'3","last_deep_scrub_stamp":"2024-09-22T01:17:38.280677+0000","last_clean_scrub_stamp":"2024-09-22T01:17:38.280677+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:44:20.842657+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"195'537","reported_seq":4637,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:39.091954+0000","last_change":"2024-09-22T01:17:39.092067+0000","last_active":"2024-09-22T01:17:39.091954+0000","last_peered":"2024-09-22T01:17:39.091954+0000","last_clean":"2024-09-22T01:17:39.091954+0000","last_became_active":"2024-09-22T01:17:18.569515+0000","last_became_peered":"2024-09-22T01:17:18.569515+0000","last_unstale":"2024-09-22T01:17:39.091954+0000","last_undegraded":"2024-09-22T01:17:39.091954+0000","last_fullsized":"2024-09-22T01:17:39.091954+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'537","last_scrub_stamp":"2024-09-22T01:17:39.091760+0000","last_deep_scrub":"195'537","last_deep_scrub_stamp":"2024-09-22T01:17:39.091760+0000","last_clean_scrub_stamp":"2024-09-22T01:17:39.091760+0000","objects_scrubbed":22,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:29.812695+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0020388070000000001,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":4385,"num_read_kb":28294,"num_write":507,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":3974613,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.502506+0000","last_change":"2024-09-22T01:17:34.502563+0000","last_active":"2024-09-22T01:17:34.502506+0000","last_peered":"2024-09-22T01:17:34.502506+0000","last_clean":"2024-09-22T01:17:34.502506+0000","last_became_active":"2024-09-22T01:17:14.554246+0000","last_became_peered":"2024-09-22T01:17:14.554246+0000","last_unstale":"2024-09-22T01:17:34.502506+0000","last_undegraded":"2024-09-22T01:17:34.502506+0000","last_fullsized":"2024-09-22T01:17:34.502506+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:34.502274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:34.502274+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.502274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:09:54.796971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"195'591","reported_seq":7060,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:37.710456+0000","last_change":"2024-09-22T01:17:37.710648+0000","last_active":"2024-09-22T01:17:37.710456+0000","last_peered":"2024-09-22T01:17:37.710456+0000","last_clean":"2024-09-22T01:17:37.710456+0000","last_became_active":"2024-09-22T01:17:09.810372+0000","last_became_peered":"2024-09-22T01:17:09.810372+0000","last_unstale":"2024-09-22T01:17:37.710456+0000","last_undegraded":"2024-09-22T01:17:37.710456+0000","last_fullsized":"2024-09-22T01:17:37.710456+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'591","last_scrub_stamp":"2024-09-22T01:17:37.710175+0000","last_deep_scrub":"195'591","last_deep_scrub_stamp":"2024-09-22T01:17:37.710175+0000","last_clean_scrub_stamp":"2024-09-22T01:17:37.710175+0000","objects_scrubbed":15,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:11:52.726569+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0027486500000000001,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8159,"num_read_kb":17265,"num_write":520,"num_write_kb":11627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":6002227,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":350,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.309617+0000","last_change":"2024-09-22T01:17:33.309640+0000","last_active":"2024-09-22T01:17:33.309617+0000","last_peered":"2024-09-22T01:17:33.309617+0000","last_clean":"2024-09-22T01:17:33.309617+0000","last_became_active":"2024-09-22T01:14:10.536202+0000","last_became_peered":"2024-09-22T01:14:10.536202+0000","last_unstale":"2024-09-22T01:17:33.309617+0000","last_undegraded":"2024-09-22T01:17:33.309617+0000","last_fullsized":"2024-09-22T01:17:33.309617+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-22T01:17:33.309530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-22T01:17:33.309530+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.309530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:14:53.503906+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"195'695","reported_seq":6609,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.743157+0000","last_change":"2024-09-22T01:17:34.743289+0000","last_active":"2024-09-22T01:17:34.743157+0000","last_peered":"2024-09-22T01:17:34.743157+0000","last_clean":"2024-09-22T01:17:34.743157+0000","last_became_active":"2024-09-22T01:15:59.611223+0000","last_became_peered":"2024-09-22T01:15:59.611223+0000","last_unstale":"2024-09-22T01:17:34.743157+0000","last_undegraded":"2024-09-22T01:17:34.743157+0000","last_fullsized":"2024-09-22T01:17:34.743157+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'695","last_scrub_stamp":"2024-09-22T01:17:34.742966+0000","last_deep_scrub":"195'695","last_deep_scrub_stamp":"2024-09-22T01:17:34.742966+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.742966+0000","objects_scrubbed":22,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:16:33.610916+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0.0032504629999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":8317,"num_read_kb":25995,"num_write":710,"num_write_kb":9999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":6117287,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"195'2","reported_seq":373,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:41.296337+0000","last_change":"2024-09-22T01:17:41.296403+0000","last_active":"2024-09-22T01:17:41.296337+0000","last_peered":"2024-09-22T01:17:41.296337+0000","last_clean":"2024-09-22T01:17:41.296337+0000","last_became_active":"2024-09-22T01:14:38.550777+0000","last_became_peered":"2024-09-22T01:14:38.550777+0000","last_unstale":"2024-09-22T01:17:41.296337+0000","last_undegraded":"2024-09-22T01:17:41.296337+0000","last_fullsized":"2024-09-22T01:17:41.296337+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'2","last_scrub_stamp":"2024-09-22T01:17:41.296139+0000","last_deep_scrub":"195'2","last_deep_scrub_stamp":"2024-09-22T01:17:41.296139+0000","last_clean_scrub_stamp":"2024-09-22T01:17:41.296139+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:31:12.854273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"195'119","reported_seq":524,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:31.128324+0000","last_change":"2024-09-22T01:17:31.128436+0000","last_active":"2024-09-22T01:17:31.128324+0000","last_peered":"2024-09-22T01:17:31.128324+0000","last_clean":"2024-09-22T01:17:31.128324+0000","last_became_active":"2024-09-22T01:13:51.177227+0000","last_became_peered":"2024-09-22T01:13:51.177227+0000","last_unstale":"2024-09-22T01:17:31.128324+0000","last_undegraded":"2024-09-22T01:17:31.128324+0000","last_fullsized":"2024-09-22T01:17:31.128324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2024-09-22T01:17:31.128115+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2024-09-22T01:17:31.128115+0000","last_clean_scrub_stamp":"2024-09-22T01:17:31.128115+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:49.214663+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"195'526","reported_seq":4329,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:38.656617+0000","last_change":"2024-09-22T01:17:38.656711+0000","last_active":"2024-09-22T01:17:38.656617+0000","last_peered":"2024-09-22T01:17:38.656617+0000","last_clean":"2024-09-22T01:17:38.656617+0000","last_became_active":"2024-09-22T01:17:18.570050+0000","last_became_peered":"2024-09-22T01:17:18.570050+0000","last_unstale":"2024-09-22T01:17:38.656617+0000","last_undegraded":"2024-09-22T01:17:38.656617+0000","last_fullsized":"2024-09-22T01:17:38.656617+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'526","last_scrub_stamp":"2024-09-22T01:17:38.656426+0000","last_deep_scrub":"195'526","last_deep_scrub_stamp":"2024-09-22T01:17:38.656426+0000","last_clean_scrub_stamp":"2024-09-22T01:17:38.656426+0000","objects_scrubbed":13,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:25:47.831345+0000","scrub_duration":19,"objects_trimmed":2,"snaptrim_duration":0.0080647840000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3963,"num_read_kb":17510,"num_write":367,"num_write_kb":4207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":2480154,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"195'1024","reported_seq":19445,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:32.315324+0000","last_change":"2024-09-22T01:17:32.315420+0000","last_active":"2024-09-22T01:17:32.315324+0000","last_peered":"2024-09-22T01:17:32.315324+0000","last_clean":"2024-09-22T01:17:32.315324+0000","last_became_active":"2024-09-22T01:14:10.535438+0000","last_became_peered":"2024-09-22T01:14:10.535438+0000","last_unstale":"2024-09-22T01:17:32.315324+0000","last_undegraded":"2024-09-22T01:17:32.315324+0000","last_fullsized":"2024-09-22T01:17:32.315324+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1024","last_scrub_stamp":"2024-09-22T01:17:32.315213+0000","last_deep_scrub":"195'1024","last_deep_scrub_stamp":"2024-09-22T01:17:32.315213+0000","last_clean_scrub_stamp":"2024-09-22T01:17:32.315213+0000","objects_scrubbed":21,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:55:15.740805+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.002782367,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":24290,"num_read_kb":81206,"num_write":967,"num_write_kb":23109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"195'725","reported_seq":4606,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:37.312395+0000","last_change":"2024-09-22T01:17:37.312478+0000","last_active":"2024-09-22T01:17:37.312395+0000","last_peered":"2024-09-22T01:17:37.312395+0000","last_clean":"2024-09-22T01:17:37.312395+0000","last_became_active":"2024-09-22T01:14:10.540981+0000","last_became_peered":"2024-09-22T01:14:10.540981+0000","last_unstale":"2024-09-22T01:17:37.312395+0000","last_undegraded":"2024-09-22T01:17:37.312395+0000","last_fullsized":"2024-09-22T01:17:37.312395+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'725","last_scrub_stamp":"2024-09-22T01:17:37.312208+0000","last_deep_scrub":"195'725","last_deep_scrub_stamp":"2024-09-22T01:17:37.312208+0000","last_clean_scrub_stamp":"2024-09-22T01:17:37.312208+0000","objects_scrubbed":13,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:33.732385+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.003406933,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":7930,"num_read_kb":12410,"num_write":657,"num_write_kb":8475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"195'586","reported_seq":4990,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:35.505308+0000","last_change":"2024-09-22T01:17:35.505399+0000","last_active":"2024-09-22T01:17:35.505308+0000","last_peered":"2024-09-22T01:17:35.505308+0000","last_clean":"2024-09-22T01:17:35.505308+0000","last_became_active":"2024-09-22T01:17:19.565378+0000","last_became_peered":"2024-09-22T01:17:19.565378+0000","last_unstale":"2024-09-22T01:17:35.505308+0000","last_undegraded":"2024-09-22T01:17:35.505308+0000","last_fullsized":"2024-09-22T01:17:35.505308+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'586","last_scrub_stamp":"2024-09-22T01:17:35.505110+0000","last_deep_scrub":"195'586","last_deep_scrub_stamp":"2024-09-22T01:17:35.505110+0000","last_clean_scrub_stamp":"2024-09-22T01:17:35.505110+0000","objects_scrubbed":11,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:18:46.458324+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0016234979999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5545,"num_read_kb":18026,"num_write":663,"num_write_kb":9242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":7662211,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"195'727","reported_seq":6355,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.600260+0000","last_change":"2024-09-22T01:17:34.600354+0000","last_active":"2024-09-22T01:17:34.600260+0000","last_peered":"2024-09-22T01:17:34.600260+0000","last_clean":"2024-09-22T01:17:34.600260+0000","last_became_active":"2024-09-22T01:17:18.569899+0000","last_became_peered":"2024-09-22T01:17:18.569899+0000","last_unstale":"2024-09-22T01:17:34.600260+0000","last_undegraded":"2024-09-22T01:17:34.600260+0000","last_fullsized":"2024-09-22T01:17:34.600260+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'727","last_scrub_stamp":"2024-09-22T01:17:34.600070+0000","last_deep_scrub":"195'727","last_deep_scrub_stamp":"2024-09-22T01:17:34.600070+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.600070+0000","objects_scrubbed":20,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:23:36.572565+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0014494169999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":5676,"num_read_kb":28875,"num_write":749,"num_write_kb":10044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4376034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"195'706","reported_seq":3079,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:40.297595+0000","last_change":"2024-09-22T01:17:40.297691+0000","last_active":"2024-09-22T01:17:40.297595+0000","last_peered":"2024-09-22T01:17:40.297595+0000","last_clean":"2024-09-22T01:17:40.297595+0000","last_became_active":"2024-09-22T01:14:10.541098+0000","last_became_peered":"2024-09-22T01:14:10.541098+0000","last_unstale":"2024-09-22T01:17:40.297595+0000","last_undegraded":"2024-09-22T01:17:40.297595+0000","last_fullsized":"2024-09-22T01:17:40.297595+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'706","last_scrub_stamp":"2024-09-22T01:17:40.297396+0000","last_deep_scrub":"195'706","last_deep_scrub_stamp":"2024-09-22T01:17:40.297396+0000","last_clean_scrub_stamp":"2024-09-22T01:17:40.297396+0000","objects_scrubbed":19,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:22:05.498871+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0017937319999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4041,"num_read_kb":12293,"num_write":854,"num_write_kb":6531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"195'909","reported_seq":4173,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:32.679900+0000","last_change":"2024-09-22T01:17:32.679948+0000","last_active":"2024-09-22T01:17:32.679900+0000","last_peered":"2024-09-22T01:17:32.679900+0000","last_clean":"2024-09-22T01:17:32.679900+0000","last_became_active":"2024-09-22T01:14:41.577982+0000","last_became_peered":"2024-09-22T01:14:41.577982+0000","last_unstale":"2024-09-22T01:17:32.679900+0000","last_undegraded":"2024-09-22T01:17:32.679900+0000","last_fullsized":"2024-09-22T01:17:32.679900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'909","last_scrub_stamp":"2024-09-22T01:17:32.679824+0000","last_deep_scrub":"195'909","last_deep_scrub_stamp":"2024-09-22T01:17:32.679824+0000","last_clean_scrub_stamp":"2024-09-22T01:17:32.679824+0000","objects_scrubbed":13,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:30:08.325563+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0017810510000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5464,"num_read_kb":21492,"num_write":1110,"num_write_kb":11698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2969274,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"195'729","reported_seq":4128,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:31.609032+0000","last_change":"2024-09-22T01:17:31.609088+0000","last_active":"2024-09-22T01:17:31.609032+0000","last_peered":"2024-09-22T01:17:31.609032+0000","last_clean":"2024-09-22T01:17:31.609032+0000","last_became_active":"2024-09-22T01:14:04.509388+0000","last_became_peered":"2024-09-22T01:14:04.509388+0000","last_unstale":"2024-09-22T01:17:31.609032+0000","last_undegraded":"2024-09-22T01:17:31.609032+0000","last_fullsized":"2024-09-22T01:17:31.609032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'729","last_scrub_stamp":"2024-09-22T01:17:31.608964+0000","last_deep_scrub":"195'729","last_deep_scrub_stamp":"2024-09-22T01:17:31.608964+0000","last_clean_scrub_stamp":"2024-09-22T01:17:31.608964+0000","objects_scrubbed":16,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:57:54.509493+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0027874610000000002,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4990,"num_read_kb":23207,"num_write":826,"num_write_kb":11300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"195'678","reported_seq":3881,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:33.175403+0000","last_change":"2024-09-22T01:17:33.175476+0000","last_active":"2024-09-22T01:17:33.175403+0000","last_peered":"2024-09-22T01:17:33.175403+0000","last_clean":"2024-09-22T01:17:33.175403+0000","last_became_active":"2024-09-22T01:14:04.516298+0000","last_became_peered":"2024-09-22T01:14:04.516298+0000","last_unstale":"2024-09-22T01:17:33.175403+0000","last_undegraded":"2024-09-22T01:17:33.175403+0000","last_fullsized":"2024-09-22T01:17:33.175403+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'678","last_scrub_stamp":"2024-09-22T01:17:33.175307+0000","last_deep_scrub":"195'678","last_deep_scrub_stamp":"2024-09-22T01:17:33.175307+0000","last_clean_scrub_stamp":"2024-09-22T01:17:33.175307+0000","objects_scrubbed":17,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:31:59.018797+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001081706,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":4864,"num_read_kb":15909,"num_write":694,"num_write_kb":5268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"195'964","reported_seq":5343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:43.388997+0000","last_change":"2024-09-22T01:17:43.389083+0000","last_active":"2024-09-22T01:17:43.388997+0000","last_peered":"2024-09-22T01:17:43.388997+0000","last_clean":"2024-09-22T01:17:43.388997+0000","last_became_active":"2024-09-22T01:14:41.579241+0000","last_became_peered":"2024-09-22T01:14:41.579241+0000","last_unstale":"2024-09-22T01:17:43.388997+0000","last_undegraded":"2024-09-22T01:17:43.388997+0000","last_fullsized":"2024-09-22T01:17:43.388997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'964","last_scrub_stamp":"2024-09-22T01:17:43.388807+0000","last_deep_scrub":"195'964","last_deep_scrub_stamp":"2024-09-22T01:17:43.388807+0000","last_clean_scrub_stamp":"2024-09-22T01:17:43.388807+0000","objects_scrubbed":14,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:20:47.746246+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0026850680000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7934,"num_read_kb":51087,"num_write":1122,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":6638830,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"195'828","reported_seq":5346,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:34.802664+0000","last_change":"2024-09-22T01:17:34.802760+0000","last_active":"2024-09-22T01:17:34.802664+0000","last_peered":"2024-09-22T01:17:34.802664+0000","last_clean":"2024-09-22T01:17:34.802664+0000","last_became_active":"2024-09-22T01:17:13.848686+0000","last_became_peered":"2024-09-22T01:17:13.848686+0000","last_unstale":"2024-09-22T01:17:34.802664+0000","last_undegraded":"2024-09-22T01:17:34.802664+0000","last_fullsized":"2024-09-22T01:17:34.802664+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'828","last_scrub_stamp":"2024-09-22T01:17:34.802482+0000","last_deep_scrub":"195'828","last_deep_scrub_stamp":"2024-09-22T01:17:34.802482+0000","last_clean_scrub_stamp":"2024-09-22T01:17:34.802482+0000","objects_scrubbed":19,"log_size":828,"log_dups_size":0,"ondisk_log_size":828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:56.553215+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0.0033349959999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":6860,"num_read_kb":26889,"num_write":933,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"195'800","reported_seq":6075,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-09-22T01:17:36.359923+0000","last_change":"2024-09-22T01:17:36.360032+0000","last_active":"2024-09-22T01:17:36.359923+0000","last_peered":"2024-09-22T01:17:36.359923+0000","last_clean":"2024-09-22T01:17:36.359923+0000","last_became_active":"2024-09-22T01:14:10.540557+0000","last_became_peered":"2024-09-22T01:14:10.540557+0000","last_unstale":"2024-09-22T01:17:36.359923+0000","last_undegraded":"2024-09-22T01:17:36.359923+0000","last_fullsized":"2024-09-22T01:17:36.359923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'800","last_scrub_stamp":"2024-09-22T01:17:36.359727+0000","last_deep_scrub":"195'800","last_deep_scrub_stamp":"2024-09-22T01:17:36.359727+0000","last_clean_scrub_stamp":"2024-09-22T01:17:36.359727+0000","objects_scrubbed":20,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:49:12.024144+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.0018188189999999999,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":7751,"num_read_kb":33135,"num_write":783,"num_write_kb":7642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":533,"num_object_clones":0,"num_object_copies":1066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":533,"num_whiteouts":529,"num_read":271190,"num_read_kb":970742,"num_write":23780,"num_write_kb":377571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":92954295,"num_keys_recovered":234,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22397,"ondisk_log_size":22397,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29316,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342524,"statfs":{"total":96636764160,"available":96606744576,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51528,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320312,"statfs":{"total":96636764160,"available":96583999488,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509495,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2351104,"data_stored":1130635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64004,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307836,"statfs":{"total":96636764160,"available":96571224064,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56268,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315572,"statfs":{"total":96636764160,"available":96579145728,"internally_reserved":0,"allocated":2945024,"data_stored":1721003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2347008,"data_stored":1130616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-22T01:17:54.541 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-22T01:17:54.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-22T01:17:54.869+0000 7f1ea7fff640 0 --2- 172.21.15.96:0/1248316601 >> [v2:172.21.15.96:3301/0,v1:172.21.15.96:6790/0] conn(0x7f1ea80a8df0 0x7f1ea813db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-22T01:17:55.103 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-22T01:17:55.104 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-22T01:17:55.104 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-22T01:17:55.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:55.175 INFO:tasks.ceph.osd.0:Stopped 2024-09-22T01:17:55.176 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-22T01:17:55.176 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:55.273 INFO:tasks.ceph.osd.1:Stopped 2024-09-22T01:17:55.273 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-22T01:17:55.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:55.353 INFO:tasks.ceph.osd.2:Stopped 2024-09-22T01:17:55.353 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-22T01:17:55.353 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:55.429 INFO:tasks.ceph.osd.3:Stopped 2024-09-22T01:17:55.430 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-22T01:17:55.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:55.497 INFO:tasks.ceph.osd.4:Stopped 2024-09-22T01:17:55.497 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-22T01:17:55.497 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:17:59.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-22T01:18:01.599 INFO:tasks.ceph.osd.5:Stopped 2024-09-22T01:18:01.599 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-22T01:18:01.599 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.673 INFO:tasks.ceph.osd.6:Stopped 2024-09-22T01:18:01.673 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-22T01:18:01.673 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.758 INFO:tasks.ceph.osd.7:Stopped 2024-09-22T01:18:01.758 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-22T01:18:01.759 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-22T01:18:01.759 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.786 INFO:tasks.ceph.mgr.y:Stopped 2024-09-22T01:18:01.786 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-22T01:18:01.786 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.816 INFO:tasks.ceph.mgr.x:Stopped 2024-09-22T01:18:01.816 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-22T01:18:01.817 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-22T01:18:01.817 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.845 INFO:tasks.ceph.mon.a:Stopped 2024-09-22T01:18:01.845 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-22T01:18:01.845 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.911 INFO:tasks.ceph.mon.c:Stopped 2024-09-22T01:18:01.911 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-22T01:18:01.911 INFO:teuthology.orchestra.run:waiting for 300 2024-09-22T01:18:01.931 INFO:tasks.ceph.mon.b:Stopped 2024-09-22T01:18:01.931 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-22T01:18:01.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-22T01:18:01.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:18:01.973 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-22T01:18:02.309 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:18:02.309 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-22T01:18:02.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:18:02.518 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-22T01:18:02.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:18:02.701 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-22T01:18:02.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:18:02.884 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-22T01:18:03.224 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:18:03.224 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-22T01:18:03.415 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:18:03.415 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-22T01:18:03.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:18:03.599 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-22T01:18:03.772 INFO:tasks.ceph:Archiving mon data... 2024-09-22T01:18:03.773 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/data/mon.a.tgz 2024-09-22T01:18:03.774 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-22T01:18:03.779 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.KjXih1e6n4 2024-09-22T01:18:03.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.KjXih1e6n4 2024-09-22T01:18:03.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.KjXih1e6n4 2024-09-22T01:18:03.987 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.KjXih1e6n4 is 421KB 2024-09-22T01:18:04.046 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.KjXih1e6n4 2024-09-22T01:18:04.052 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/data/mon.c.tgz 2024-09-22T01:18:04.052 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-22T01:18:04.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-22T01:18:04.100 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.l2xO3X88r4 2024-09-22T01:18:04.101 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.l2xO3X88r4 2024-09-22T01:18:04.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.l2xO3X88r4 2024-09-22T01:18:04.344 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.l2xO3X88r4 is 570KB 2024-09-22T01:18:04.402 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.l2xO3X88r4 2024-09-22T01:18:04.407 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/data/mon.b.tgz 2024-09-22T01:18:04.407 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-22T01:18:04.413 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.M0TxSwrGF8 2024-09-22T01:18:04.413 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.M0TxSwrGF8 2024-09-22T01:18:04.592 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.M0TxSwrGF8 2024-09-22T01:18:04.654 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.M0TxSwrGF8 is 570KB 2024-09-22T01:18:04.716 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.M0TxSwrGF8 2024-09-22T01:18:04.721 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-22T01:18:04.721 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T01:18:04.723 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-22T01:18:04.780 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T01:18:04.818 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-22T01:18:04.819 INFO:tasks.ceph:Archiving crash dumps... 2024-09-22T01:18:04.820 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi096/crash 2024-09-22T01:18:04.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T01:18:04.837 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi150/crash 2024-09-22T01:18:04.838 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-22T01:18:04.873 INFO:tasks.ceph:Compressing logs... 2024-09-22T01:18:04.873 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T01:18:04.881 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-22T01:18:04.907 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-09-22T01:18:04.908 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-09-22T01:18:04.908 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 2024-09-22T01:18:04.908 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-client.admin.43126.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36344.log 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39935.log 2024-09-22T01:18:04.909 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-09-22T01:18:04.910 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36344.log.gz 2024-09-22T01:18:04.910 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32099.log 2024-09-22T01:18:04.910 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39935.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39935.log.gz -5 2024-09-22T01:18:04.910 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42944.log 2024-09-22T01:18:04.911 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42918.log 2024-09-22T01:18:04.911 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32099.log.gz 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34802.log 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42944.log.gz 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42918.log.gz 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41878.log 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41904.log 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34802.log.gz 2024-09-22T01:18:04.912 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2024-09-22T01:18:04.913 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41878.log.gz 2024-09-22T01:18:04.913 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41904.log.gz 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38799.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-09-22T01:18:04.914 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40926.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32141.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40926.log.gz 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32775.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38000.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32141.log.gz 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42710.log 2024-09-22T01:18:04.916 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32775.log.gz 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35749.log 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38000.log.gz 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35962.log 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42710.log.gz 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35749.log.gz 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34472.log 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35962.log: /var/log/ceph/ceph-client.admin.32151.log 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35962.log.gz 2024-09-22T01:18:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34472.log.gz 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32151.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32151.log.gz 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35913.log 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-09-22T01:18:04.920 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-09-22T01:18:04.924 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-09-22T01:18:04.924 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35913.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34337.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32349.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34337.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37302.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32349.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41800.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37302.log.gz 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32136.log 2024-09-22T01:18:04.925 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41800.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32360.log 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32136.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36189.log 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32360.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41092.log 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36189.log.gz 2024-09-22T01:18:04.926 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32801.log 2024-09-22T01:18:04.931 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38444.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41092.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32568.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32801.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38444.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42970.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35105.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32568.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33727.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35105.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33727.log.gz 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33922.log 2024-09-22T01:18:04.932 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32566.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/ceph-client.admin.33922.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33922.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32027.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32566.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32152.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32027.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43204.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.32152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32073.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32152.log.gz 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34532.log 2024-09-22T01:18:04.933 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43204.log.gz 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32073.log.gz 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36488.log 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34532.log.gz 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-22T01:18:04.934 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42866.log 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41956.log 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36488.log.gz 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42195.log 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42866.log: 0.0%/var/log/ceph/ceph-client.admin.41956.log: -- replaced with /var/log/ceph/ceph-client.admin.42866.log.gz 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41956.log.gz 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-09-22T01:18:04.939 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34235.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42195.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33169.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43282.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34235.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32333.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39046.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32333.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40705.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39046.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32979.log 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2024-09-22T01:18:04.940 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38482.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40705.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33407.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32979.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41179.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38482.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33407.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41179.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34001.log 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-09-22T01:18:04.941 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34246.log 2024-09-22T01:18:04.942 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz 2024-09-22T01:18:04.949 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26540.log 2024-09-22T01:18:04.949 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26516.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26540.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26540.log.gz 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26468.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26516.log.gz 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26468.log.gz 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26420.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26372.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/ceph-client.admin.26420.log: --verbose -- /var/log/ceph/ceph-client.admin.26396.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26420.log.gz 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26372.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T01:18:04.950 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26372.log.gz 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26492.log 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26396.log.gzgzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.26444.log 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr: 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.26492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26492.log.gz 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/ceph-client.admin.26444.log: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26444.log.gz 2024-09-22T01:18:04.951 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T01:18:04.967 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38711.log 2024-09-22T01:18:04.967 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34001.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31965.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34246.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38711.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32978.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31965.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39766.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32978.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32149.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39766.log.gz 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36078.log 2024-09-22T01:18:04.968 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32149.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36078.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40500.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39572.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40500.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33521.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39572.log.gz 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40741.log 2024-09-22T01:18:04.969 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33521.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26502.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40741.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42632.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37509.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.26502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33648.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26502.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43152.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42632.log.gz 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-09-22T01:18:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33648.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43152.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32567.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37720.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32567.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37029.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37720.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37227.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-09-22T01:18:04.971 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37029.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37227.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43048.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41826.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37602.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43048.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38575.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37475.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42788.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38575.log.gz 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33769.log 2024-09-22T01:18:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37475.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37801.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42788.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35544.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33769.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37801.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35544.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35544.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35688.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35433.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.35688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34689.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35688.log.gz 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41454.log 2024-09-22T01:18:04.973 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35433.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38270.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41454.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42892.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38270.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34895.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42892.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35029.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-09-22T01:18:04.974 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34895.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40424.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35029.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32148.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40424.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32148.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32905.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32569.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32548.log 2024-09-22T01:18:04.975 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32905.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35162.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32569.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37898.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32548.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36224.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35162.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39317.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37898.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36224.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36224.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43360.log/var/log/ceph/ceph-client.admin.39317.log: 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39317.log.gz 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35376.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-09-22T01:18:04.976 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43360.log.gz 2024-09-22T01:18:04.977 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32981.log 2024-09-22T01:18:04.977 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35376.log.gz 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41051.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38005.log 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36435.log 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32981.log.gz/var/log/ceph/ceph-client.admin.38005.log: 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43178.log 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32347.log 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36435.log: /var/log/ceph/ceph-client.admin.43178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36435.log.gz 2024-09-22T01:18:04.991 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37071.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43178.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32347.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37071.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42996.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26354.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42996.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34085.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42996.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.26354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32002.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26354.log.gz 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32549.log 2024-09-22T01:18:04.992 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34085.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32002.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35325.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32549.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35325.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32153.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34240.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33128.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32153.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34240.log: /var/log/ceph/ceph-client.admin.33128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36036.log 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34240.log.gz 2024-09-22T01:18:04.993 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33128.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36036.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37136.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38996.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37136.log.gz 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34190.log 2024-09-22T01:18:04.994 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38996.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38952.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33860.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34190.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32340.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38952.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33860.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32565.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41480.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32340.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32565.log.gz 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39199.log 2024-09-22T01:18:04.995 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41480.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42398.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34924.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39199.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37945.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42398.log: /var/log/ceph/ceph-client.admin.34924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42398.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34924.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37945.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37945.log.gz -5 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40828.log: /var/log/ceph/ceph-client.admin.42658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-09-22T01:18:04.996 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38545.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39668.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.32047.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38545.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38545.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33581.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32047.log: /var/log/ceph/ceph-client.admin.33581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32047.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33581.log.gz 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41265.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41990.log: /var/log/ceph/ceph-client.admin.40862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38159.log 2024-09-22T01:18:04.997 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-09-22T01:18:04.998 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz 2024-09-22T01:18:04.998 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41265.log.gz 2024-09-22T01:18:04.998 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-09-22T01:18:04.998 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32348.log 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38159.log: /var/log/ceph/ceph-client.admin.41748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40331.log 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38159.log.gz 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32348.log.gz 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40120.log 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33803.log 2024-09-22T01:18:05.007 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40331.log: /var/log/ceph/ceph-client.admin.40120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40331.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40120.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33803.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36885.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40997.log: /var/log/ceph/ceph-client.admin.36885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32150.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36885.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37186.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: /var/log/ceph/ceph-client.admin.33314.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32150.log: /var/log/ceph/ceph-client.admin.33314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34033.log 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32150.log.gz 2024-09-22T01:18:05.008 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33314.log.gz 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42413.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz /var/log/ceph/ceph-client.admin.33527.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39623.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34033.log: /var/log/ceph/ceph-client.admin.33527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34033.log.gz 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33527.log.gz 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39623.log.gzgzip 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41930.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35872.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32879.log: /var/log/ceph/ceph-client.admin.41930.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41930.log.gz 2024-09-22T01:18:05.009 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35872.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41376.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34845.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.41376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41376.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34845.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42008.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38895.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42008.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35187.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43100.log 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38895.log.gz 2024-09-22T01:18:05.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35187.log: /var/log/ceph/ceph-client.admin.43100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35187.log.gz 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43100.log.gz 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35574.log 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35574.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34986.log 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:05.011 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2024-09-22T01:18:05.012 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34986.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33458.log 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38929.log 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43308.log: /var/log/ceph/ceph-client.admin.33458.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33458.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36972.log 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31073.log 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38929.log.gz 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36972.log.gzgzip 2024-09-22T01:18:05.017 INFO:teuthology.orchestra.run.smithi096.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31073.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32853.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42405.log: /var/log/ceph/ceph-client.admin.36130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32853.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39091.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39091.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz 2024-09-22T01:18:05.018 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-09-22T01:18:05.019 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38629.log 2024-09-22T01:18:05.019 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.42209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38629.log: /var/log/ceph/ceph-client.admin.41130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38629.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36791.log 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32968.log 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36791.log.gz 2024-09-22T01:18:05.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-09-22T01:18:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.0.32968.log: 95.5% -- replaced with /var/log/ceph/ceph-client.0.32968.log.gz 2024-09-22T01:18:05.022 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2024-09-22T01:18:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-22T01:18:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-09-22T01:18:05.023 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42002.log 2024-09-22T01:18:05.024 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-22T01:18:05.024 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-22T01:18:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37437.log 2024-09-22T01:18:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42002.log: /var/log/ceph/ceph-client.admin.42198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42002.log.gz 2024-09-22T01:18:05.025 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-22T01:18:05.025 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35289.log 2024-09-22T01:18:05.026 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-09-22T01:18:05.026 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37437.log.gz 2024-09-22T01:18:05.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35289.log.gz 2024-09-22T01:18:05.027 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33343.log 2024-09-22T01:18:05.027 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-09-22T01:18:05.028 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41269.log 2024-09-22T01:18:05.028 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36701.log 2024-09-22T01:18:05.028 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33343.log: /var/log/ceph/ceph-client.admin.41269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33343.log.gz 2024-09-22T01:18:05.028 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41269.log.gz 2024-09-22T01:18:05.028 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40067.log 2024-09-22T01:18:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36578.log 2024-09-22T01:18:05.029 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36701.log: /var/log/ceph/ceph-client.admin.40067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36701.log.gz 2024-09-22T01:18:05.030 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40067.log.gz 2024-09-22T01:18:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32749.log 2024-09-22T01:18:05.030 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-09-22T01:18:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36578.log: /var/log/ceph/ceph-client.admin.32749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36578.log.gz 2024-09-22T01:18:05.031 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32749.log.gz 2024-09-22T01:18:05.031 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32935.log 2024-09-22T01:18:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-09-22T01:18:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43022.log: /var/log/ceph/ceph-client.admin.32935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-09-22T01:18:05.032 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32935.log.gz 2024-09-22T01:18:05.032 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43413.log 2024-09-22T01:18:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-09-22T01:18:05.033 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37277.log 2024-09-22T01:18:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21501.log 2024-09-22T01:18:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43413.log: /var/log/ceph/ceph-client.admin.37277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43413.log.gz 2024-09-22T01:18:05.034 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37277.log.gz 2024-09-22T01:18:05.034 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35495.log 2024-09-22T01:18:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40559.log 2024-09-22T01:18:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.tmp-client.admin.21501.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21501.log.gz 2024-09-22T01:18:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.35495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35495.log.gz 2024-09-22T01:18:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38373.log 2024-09-22T01:18:05.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-09-22T01:18:05.036 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32361.log 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38373.log.gz 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32361.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32361.log.gz 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37388.log 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-09-22T01:18:05.037 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40014.log 2024-09-22T01:18:05.038 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37598.log 2024-09-22T01:18:05.038 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37388.log.gz 2024-09-22T01:18:05.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40014.log.gz 2024-09-22T01:18:05.039 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33954.log 2024-09-22T01:18:05.039 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.37598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37598.log.gz 2024-09-22T01:18:05.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-09-22T01:18:05.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42005.log 2024-09-22T01:18:05.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33954.log.gz 2024-09-22T01:18:05.041 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-09-22T01:18:05.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-09-22T01:18:05.042 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42005.log: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.42005.log.gz 2024-09-22T01:18:05.042 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32827.log 2024-09-22T01:18:05.042 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39812.log 2024-09-22T01:18:05.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-09-22T01:18:05.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32827.log.gz 2024-09-22T01:18:05.043 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-22T01:18:05.043 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39812.log.gz 2024-09-22T01:18:05.044 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34470.log 2024-09-22T01:18:05.045 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32541.log 2024-09-22T01:18:05.045 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34470.log: /var/log/ceph/ceph-client.admin.41511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34470.log.gz 2024-09-22T01:18:05.045 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz 2024-09-22T01:18:05.045 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-09-22T01:18:05.046 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40382.log 2024-09-22T01:18:05.046 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32541.log.gz 2024-09-22T01:18:05.046 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-09-22T01:18:05.046 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:05.047 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40382.log.gz 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38678.log 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42010.log 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38678.log.gz 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42414.log 2024-09-22T01:18:05.048 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42010.log.gz 2024-09-22T01:18:05.050 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42606.log 2024-09-22T01:18:05.050 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33582.log 2024-09-22T01:18:05.050 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2024-09-22T01:18:05.050 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42606.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38311.log 2024-09-22T01:18:05.050 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42606.log.gz 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33582.log.gz 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26614.log 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34035.log 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38311.log.gz 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.26614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26614.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-09-22T01:18:05.052 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:05.053 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34035.log.gz 2024-09-22T01:18:05.053 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38094.log 2024-09-22T01:18:05.053 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32353.log 2024-09-22T01:18:05.053 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.43386.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-09-22T01:18:05.054 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38094.log.gz 2024-09-22T01:18:05.054 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38211.log 2024-09-22T01:18:05.054 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.32353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32353.log.gz 2024-09-22T01:18:05.054 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-09-22T01:18:05.055 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42814.log 2024-09-22T01:18:05.055 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.38211.log: /var/log/ceph/ceph-client.admin.42486.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38211.log.gz 2024-09-22T01:18:05.055 INFO:teuthology.orchestra.run.smithi096.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-09-22T01:18:05.055 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40612.log 2024-09-22T01:18:05.056 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41618.log 2024-09-22T01:18:05.057 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.42814.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42814.log.gz 2024-09-22T01:18:05.057 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.40612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40612.log.gz 2024-09-22T01:18:05.057 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34144.log 2024-09-22T01:18:05.057 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41618.log.gz 2024-09-22T01:18:05.057 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36881.log 2024-09-22T01:18:05.058 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39401.log 2024-09-22T01:18:05.058 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.34144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34144.log.gz 2024-09-22T01:18:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.36881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36881.log.gz 2024-09-22T01:18:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-09-22T01:18:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.39401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39401.log.gz 2024-09-22T01:18:05.059 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43074.log 2024-09-22T01:18:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33220.log 2024-09-22T01:18:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.41537.log: /var/log/ceph/ceph-client.admin.43074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-09-22T01:18:05.060 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43074.log.gz 2024-09-22T01:18:05.060 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42208.log 2024-09-22T01:18:05.061 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.33220.log: 0.0%/var/log/ceph/ceph-client.admin.42208.log: -- replaced with /var/log/ceph/ceph-client.admin.33220.log.gz 2024-09-22T01:18:05.061 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42208.log.gz 2024-09-22T01:18:05.153 INFO:teuthology.orchestra.run.smithi150.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-22T01:18:05.206 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-22T01:18:05.245 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-22T01:18:06.191 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-22T01:18:06.846 INFO:teuthology.orchestra.run.smithi150.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-22T01:18:07.470 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-22T01:18:07.781 INFO:teuthology.orchestra.run.smithi096.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-22T01:18:07.968 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-22T01:18:07.970 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-22T01:18:07.970 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m3.046s 2024-09-22T01:18:07.970 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m8.636s 2024-09-22T01:18:07.970 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.375s 2024-09-22T01:18:08.115 INFO:teuthology.orchestra.run.smithi096.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-22T01:18:08.480 INFO:teuthology.orchestra.run.smithi096.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-22T01:18:11.454 INFO:teuthology.orchestra.run.smithi096.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-22T01:18:11.455 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-22T01:18:11.455 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m6.572s 2024-09-22T01:18:11.455 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m16.157s 2024-09-22T01:18:11.455 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.751s 2024-09-22T01:18:11.456 INFO:tasks.ceph:Archiving logs... 2024-09-22T01:18:11.456 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi096/log 2024-09-22T01:18:11.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T01:18:13.236 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi150/log 2024-09-22T01:18:13.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-22T01:18:14.034 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-22T01:18:14.042 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-22T01:18:14.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T01:18:14.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-22T01:18:14.117 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-22T01:18:14.125 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-22T01:18:14.125 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:18:14.127 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-22T01:18:14.138 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:18:14.139 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-22T01:18:14.139 INFO:teuthology.orchestra.run.smithi096.stdout:+hv01.front.sepi 67.205.162.81 3 u 29 64 177 0.105 -0.849 0.747 2024-09-22T01:18:14.139 INFO:teuthology.orchestra.run.smithi096.stdout:+hv02.front.sepi 63.231.80.2 3 u 26 64 177 0.212 +0.322 0.696 2024-09-22T01:18:14.139 INFO:teuthology.orchestra.run.smithi096.stdout:*hv03.front.sepi 74.6.168.72 3 u 35 64 177 0.099 -1.171 0.820 2024-09-22T01:18:14.139 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:18:14.142 INFO:teuthology.orchestra.run.smithi150.stdout: remote refid st t when poll reach delay offset jitter 2024-09-22T01:18:14.142 INFO:teuthology.orchestra.run.smithi150.stdout:============================================================================== 2024-09-22T01:18:14.142 INFO:teuthology.orchestra.run.smithi150.stdout:+hv01.front.sepi 67.205.162.81 3 u 28 64 177 0.073 -2.726 1.697 2024-09-22T01:18:14.143 INFO:teuthology.orchestra.run.smithi150.stdout:+hv02.front.sepi 63.231.80.2 3 u 27 64 177 0.065 -0.043 1.214 2024-09-22T01:18:14.143 INFO:teuthology.orchestra.run.smithi150.stdout:*hv03.front.sepi 74.6.168.72 3 u 32 64 177 0.069 -0.349 2.138 2024-09-22T01:18:14.143 INFO:teuthology.orchestra.run.smithi150.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-22T01:18:14.143 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-22T01:18:14.151 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-22T01:18:14.151 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-22T01:18:14.160 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-22T01:18:14.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-22T01:18:14.177 INFO:teuthology.task.internal:Duration was 881.731122 seconds 2024-09-22T01:18:14.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-22T01:18:14.185 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-22T01:18:14.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T01:18:14.187 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-22T01:18:14.223 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-22T01:18:14.223 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-22T01:18:14.223 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T01:18:14.277 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-22T01:18:14.277 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-22T01:18:14.288 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-22T01:18:14.288 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T01:18:14.325 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-22T01:18:14.358 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-22T01:18:14.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T01:18:14.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-22T01:18:14.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-22T01:18:14.482 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-22T01:18:14.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T01:18:14.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-22T01:18:14.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-22T01:18:14.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:18:14.537 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:18:14.546 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-22T01:18:14.557 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-22T01:18:14.579 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:18:14.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:18:14.612 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-22T01:18:14.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-22T01:18:14.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-22T01:18:14.635 INFO:teuthology.task.internal:Transferring archived files... 2024-09-22T01:18:14.635 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi096 2024-09-22T01:18:14.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T01:18:14.682 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914605/remote/smithi150 2024-09-22T01:18:14.682 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-22T01:18:14.703 INFO:teuthology.task.internal:Removing archive directory... 2024-09-22T01:18:14.703 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T01:18:14.722 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-22T01:18:14.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-22T01:18:14.758 INFO:teuthology.task.internal:Not uploading archives. 2024-09-22T01:18:14.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-22T01:18:14.766 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-22T01:18:14.767 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T01:18:14.769 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-22T01:18:14.772 INFO:teuthology.orchestra.run.smithi096.stdout: 655429 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 01:18 /home/ubuntu/cephtest 2024-09-22T01:18:14.794 INFO:teuthology.orchestra.run.smithi150.stdout: 658614 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 22 01:18 /home/ubuntu/cephtest 2024-09-22T01:18:14.795 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-22T01:18:14.804 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-22T01:18:14.826 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-stupid supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} duration: 881.7311222553253 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-22T01:18:14.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-22T01:18:14.904 INFO:teuthology.run:pass